The world of testing is divided into camps. Those camps argue with each other, sometimes. Mostly they ignore each other. These camps are like religions of software testing. They are determined and persistent patterns of belief, speech, and behavior. They could be called paradigms. In my community, we call them “schools.”
I didn’t create the schools (only the members of each school can do that) but I helped make labels for them. The labels represent these persistent patterns. Paul Gerrard wants them to go away. This puzzles me. It’s like trying to make culture go away. You can ignore cultural differences, of course, as long as you only work within one culture. You can ignore cultural differences if you wish to remain ignorant and provincial.
I’m a pluralist. I have my own testing religion (the Context-Driven School), but I recognize that there are others in the world. Besides Context-Driven, I share the world with adherents to the Factory School, Quality Control School, Analytical School, and Test-Driven Design School, and I share the world with people who form their own schools drawn from each of the other ones.
Paul suggests that no one wants to be a member of a school and that the schools don’t align to real-life testing. Paul’s wrong about that, and profoundly so. Actually the only people NOT a member of a school are those who have no beliefs or ideas about testing. But if you test, you must have beliefs about testing. You have a vocabulary and a sense of the tasks that are to be completed. As you gain experience and education, your beliefs (and consequently your actions) will exhibit an increasingly definite pattern and that can be observed and studied. It may be, as you learn, you change your beliefs. It may be that your beliefs are contradictory, at times. Still, as you get your head together about testing, a pattern emerges. That’s what it means to “know how to test.”
It’s true that the labels I use for schools are stereotypes– if that means a generic pattern. They are useful generic patterns, because they allow me to more efficiently characterize someone’s testing practices. I can say “that’s a factory approach except for this part, here.” People who study culture do use pattern languages to aid in discussing them. This is not some crazy idea of my own.
Paul Gerrard has written quite a bit about testing. He has expressed specific ideas and his ideas stand in contrast to alternative ideas. For instance, he has argued for quantitative risk analysis in test planning, independent of project context. That attitude is consistent with both the Factory and Analytical School ways of thinking, both of which extol the trappings of science (e.g. numbers). The Factory School way is to apply numbers without worrying about whether they have a rational scientific foundation (based on my arguments with Paul, he seems to be in that camp). Factory schoolers are keen to remove the icky human element from testing, wherever possible. They are annoyed by touchy feely things such as their own human infirmity. The Analytical School way is to limit themselves to laboratory contexts where the numbers apply or trying to change projects to ft the assumptions of the numbers. Analytical schoolers are generally academics.
I have debated quantitative risk analysis with Paul Gerrard. He certainly has different beliefs and ethics about numbers than I do. Without making a claim about who is right, isn’t it reasonable to say that he and I belong to different schools of thought?
People don’t necessarily sign up for these schools. Someone could describe himself as “spontaneous and care-free” without joining a club, or committing to that way of being for all time. Similarly, I could describe someone as “methodical and studious” even if he saw himself as spontaneous and care-free. I am not forcing anyone to join an organization by describing them according to a pattern of behavior. Schools are not necessarily communities or organizations.
In the case of the Context-Driven School, there is a community. At least one: mine. You can be context-driven without being in my community, but if you are context-driven in your behavior (meaning, in short, that you reject best practices and study skills and contexts instead) then you will probably be happy to join the software-testing forum at Yahoogroups.com. That’s the forum for our community.
When we began to declare testing schools, almost ten years ago, what we hoped to do was to foster a more scientific community. In science, the notion of schools is well-established. Since the time of the ancient Greeks, this has been so. By failing to establish this in the testing industry, we foster endless unproductive debate based on the assumption that we all share one foundation. In fact, our differences are very deep. There is not even a consensus, in the testing industry, about what “testing” means!
Briefly, what are the strengths and weaknesses of each school, in your mind? For example, if an organization adheres to the Factory school (knowingly or not), what is the most important thing they could learn from another school?
[James’ Reply: Remember, schools are world-views… paradigms… like religion, they are all encompassing. One does not shop for them like one might shop for a new car. I once was part of the Quality Control School because I thought testers were really just agents of assuring quality. I now believe I was wrong about that. Therefore, I guess I could say that what I learned from the Quality Control School is that it doesn’t work. I was once a part of the Factory School, but I outgrew that, too. I have a fondness for the Analytical School, but I’m not an academic, so I have to live in a world where I must solve the problems that come to me, rather than the ones I choose. I know too much about testing to fit my mind into the Test-Driven School.
You can learn from anything.
I’m a member of the Context-Driven school because I believe in tailoring solutions to problems, and because I see those problems as mostly human problems, rather than technical ones.]
Someone, perhaps in a year or so, should point out to Mr Gerrard that when he writes, in the linked text:
Call us the ‘some of the time school’, or maybe the ‘appropriate’ school or maybe, the ‘it depends on the context’ school. Whatever. We choose to adopt appropriate practices depending on our context.
…He reminds me of some people I know. Who have, you know… something they call a school of thought like that. The prominent names will come to me in a minute, I’m sure. They’re right on the tip of my tongue. Darn it! I just saw the phrase “context-sensitive” somewhere around here and now I’ve gone and lost track.
/sarcasm
But when he then says
The schools don’t work because the proposed stereotypes do not align with the behaviours of people in real projects.
I fear he has several epistemic blind spots, not the least of which is what Bateson called “Jackdaw Epistemology”. I don’t know what to advise on that score.
What is your view on those people who say “I pick good ideas from each school and I don’t have to belong to a school to do that”. This is a kind of mixing. I am pretty sure that ideas, practices, techniques and what you do for living (not that you believe in) do not belong to any school. I am at loss to talk to these people who say – lets pick up best of every school.
Shrini
[James’ Reply: If someone has no definite idea of what they do when they test, I call that the “Oblivious Unschool.” If they do have definite ideas, then they have a school. It makes no difference whether they declare the school or name the school. To “select” from different schools is really meaningless, since practices don’t belong to schools, and because schools are self-consistent accounts of the way the world works. You can’t compose a notion of testing from different schools without constructing a new school. The only way you can legitimately claim to have no school is to be deeply confused about testing.
I didn’t make this up. I’m simply talking about the basics of culture in humans. Do you think it’s reasonable for someone to say “I don’t have a culture, myself, but I freely borrow from any culture”? I would find that unbelievable and rather silly.]
The concept of testing schools leaves me somewhat unsatisfied, since it seems that a large majority of people outside the Context-Driven school reject the basic idea of schools. But still I think that knowing about the schools can be useful.
[James’ Reply: Unless you have performed a formal survey, you aren’t in a position to say that a majority of people reject the basic notion of schools. But I bet if you surveyed sociologists, philosophers, scientists, and historians of science, you would find widespread acceptance of the school concept. It would be absurd to deny that there are different communities within a growing craft, and that those communities take different views of that craft.]
For example, on a recent project I heard people talking about testing in a fundamentally different way than what I was recommending. Though I can’t remember the names or all of the characteristics of the schools off the top of my head, I knew that I had been immersed in a school very different from my own. I determined to push to do my own testing according to the way I felt was best, but I also knew that trying to change the approach that everyone else was using in the short term would not be feasible. Since my work was loosely coupled with theirs, this worked well.
[James’ Reply: An acceptance of the notion of schools allows you to be more effectively pluralistic in your dealings with other people.]
I’d be curious to hear about progress anyone has made in the conversation between the proponents of schools and those who think it’s not a useful model. Have the doubters made any valid arguments against the testing schools, and if so, what was the response?
[James’ Reply: I’m waiting for anyone to argue against schools who has any knowledge of the history of human ideas. I don’t know where Paul Gerrard was educated, but he does not seem to have much of a liberal arts foundation. Study a bit of history, especially look into the rise and fall of scholasticism, or the debate between the Cartesians and Newtonians for the soul of physics, and you’ll see that the schools concept is hard to refute. It’s been with us for ages.]
I am again in agreement.
Schools of testing are indeed very much like religions. Whilst you may not agree with every tenant of the school, there are basic underlying principles that the members of the body within that school can all cling to and agree on. The rest of our work and the view of the way it is conducted is very much up for grabs. Whilst each school believes there are certain important preconditions for qualification into it, there are other aspect that upon which one can take different view points on.
I should clarify my statement “it seems that a large majority of people outside the Context-Driven school reject the basic idea of schools.” I meant to say “it seems that a large majority of testers outside the Context-Driven school reject the basic idea of schools of testing.” Or more to the point, I have seen very few people people who claim to be a member of any school other than the Context-Driven school.
[James’ Reply: Come on, Danny. It makes no difference whether people claim to be a member of a school. The schools concept is useful for those of us who take controversies in the craft seriously. We use it to categorize and summarize controversies. We use it to recognize each other.]
At least with religion, even if Christians say the Jews are wrong, and vice-versa, and even if some claim that the other religion is totally invalid, don’t most of them at least generally agree that they *have* a particular religion and can identify their religion in a way so everyone we can know who follows a roughly similar ideology?
[James’ Reply: I think most people in the craft are not educated enough, and perhaps don’t care enough, to understand that they have a worldview of testing, and that their worldview is not the only view.]
One can wonder if one can see different personality types adhere to different schools of testing. For example if you take the Myers-Briggs Type Indicator and correlate them on the different schools. i.e. comparing the “see the pattern” person with the “lets makes lists” person and see what schools they prefer.
James,
This is an interesting article and it’s got me wondering if particular schools often correlate to specific specialities. For example, if I was particularly adept at automated test methods but also a strong advocate of exploratory testing, does that point me in the way of one particular school?
[James’ Reply: I hope everybody chooses the context-driven school. It’s the best I know.]
Also, do you think the ability to advertise the testing school that you belong within would supercede the need for advertising what training you have? – in essence, is your school more relevant than your training?
[James’ Reply: I wouldn’t say that it’s necessarily important to advertise your school. I advertise it because I think it helps me and my clients to do so.]
James — I believe a person can BELIEVE they do not belong to a school, and just pick practices as they need them. Thinking in a cultural way, this would be like a world traveler, who has traveled so long he long ago shed the identifiable characteristics of his homeland, and travels so often that he has not picked up the identifiable characteristics of any other local. He may speak many languages fluently, and change his dress and demeanor to blend seamlessly with his environment. My fist inclination was to describe such a person as a gypsy, but gypsy’s had (have) their own culture, so the School of Gypsy Testing seems to not be accurate here. I could just describe them as being Context-Driven, which this very much describes…but I am not trying to prove my own self serving point; AND, if this person ends up testing in a culture that highly values, for example, best practices, is it truly context-driven to simply assimilate the values of the school you get dropped into? I can see the debate, as that is now your context — but that is not how I would choose to behave as a context driven tester. Perhaps the Mercenary School of Testing? Part of being a good hit man is blending in, and all you really care about is the money anyway…hmmm, that might work. I believe this is a really interesting idea, and would rather kick it around some than just dismiss it. Keep in mind, I agree with you that ultimately they land in some school based on their exhibited behavior…but, their belief is that they do not belong to a school, so that is their reality…and with no school to adhere to, what if their exhibited behavior changes so rapidly and often and dramatically that all current school definitions fail to describe them sufficiently?
Sincerely,
David
[James’ Reply: Of course, many testers would claim not to belong to a school. Even so, unless they are simply confused, their choices will follow patterns that can be productively labelled and categorized and distinguished from alternatives. Perhaps the pattern is that they pick practices based on what solves the testing problem in the situation– if so, we call that context-driven testing.
In my experience, many testing writers who claim to be pragmatic are unable or unwilling to defend the pragmatism of their behavior. I think it’s more the former than the latter, since such “pragmatists”, such as Rex Black, promote their own favorite practices without actually thinking about alternatives. (And it’s easy to call yourself a pragmatist if your only standard is you haven’t yet been killed or arrested by behaving as you do.)
If you find yourself in a context that is irrationally obsessed with certain practices, and you go along with that, then you are willfully engaging in irrationally obsessed behavior. I might then think of you as a member of the Theatrical School of testing– you believe ceremony is more important than problem-solving. That is not engineering, but people in the Theatrical School will pretend that it is. Pretense is their core value.
If you tell me that you are merely following the context of that group, and therefore that is context-driven behavior, I would agree– but it’s not context-driven testing. It’s context-driven job-keeping. Context-driven jobbers say anything to get work. You might as well fake your CV while you’re at it.
Excellent context-driven testing means studying the underlying dynamics of contexts and responses to context, so that you can see the problems that matter and solve them, even as the situation shifts beneath you. This cannot be done by promoting a narrow set of “best” practices. Of course, there are many possible variations of context-driven behavior, but I will spend my time elaborating on my own version, rather than spoon-feeding the nay-sayers. They will have to pick up a book on the history of science and do a bit of learning on their own.]
Hello James,
As a frequent reader of your and other blogs postings related to testing schools triggered me to think about them. For this I hope we keep talking about this topic as it will help defining boundaries or reshaping the already set boundaries.
Somehow I cannot accept that there are no schools of testing, neither the idea that those have to go away. The basic idea of this is the perception that to know yourselves you have to know the others. As help you can use labeling. Based on this labeling you can start discussions which should help to evolve. These discussions are needed as the next step beyond knowing is understanding.
In my opinion Evolution exists. Currently Mr. Brett Pettichord defined 5 schools of testing. This doesn’t mean in my opinion that it will stay at these 5 schools. Only those schools will not go away. Supporting the idea of ignoring schools is to me similar as a black-hole thought. If people are supporting that thought everything will be absorbed and no one can predict what is on the other side. Or predict if there is something.
I liked the comment of you about Context-driven Testing and Context-driven Job-keeping. I see some people who are adapting. Adaptation is in my opinion good. Only, it should benefit the customer, not (only) you.
Yet, the difficulty for me is still to identify which school I belong. I believe that context-driven approach is one which fits in my attitude and nature. Only there is still a long way to go to understand completely what it is all about. I have to read, write, talk, discuss and learn more about this school before I can proudly say: This is what I am, this is what I do and this is how I do it.
This leaves me to the thought about a vacuum in testing schools. To belong to a school you need understanding and practice. To get that understanding and experience you need some time. Within this timeframe you might come in a situation you shift your position within the principles of those schools. This might be a pitfall as it forms believe in you as a tester. You cannot say this moment I’m tending to this school and the next assignment you belong to another school.
One option for avoiding this vacuum I see a merging of schools principles. Were the tester defines for them selves which principles are of most of value for them. During the known timeframe the tester starts learning and tending to move towards accepting the basic principles of the defined school. As a tester starts moving I think he needs mentors for this. People who support their school of testing and by open discussions let the tester teach what school to choose. I think it is important that those mentors are not trying to convince what is best, they should give direction and let the tester defines his/her own direction.
As I’m not aware of a place were these discussions are held I appreciate the time you and others spend on weblogs like this to give me direction. Thanks.
[James’ Reply: Thanks, Jeroen. This is the kind of thinking we were hoping to spur by declaring the schools in the first place.]
If you tell me that you are merely following the context of that group, and therefore that is context-driven behavior, I would agree– but it’s not context-driven testing. It’s context-driven job-keeping. Context-driven jobbers say anything to get work. You might as well fake your CV while you’re at it.
I’ve been calling that the “DRTB School” — Don’t Rock the Boat — and I have just coined a really scurrilous name for the confluence of that plus money-hunger:
Don’t Rock The Boat [and Greed], which abbreviates of course as DRTBAG. Sssssh! It’s our little secret.
The idea of “schools of software testing” has been divisive, annoying, threatening, and useful since we first introduced it. Some of my thoughts on the history of the concept are here: https://www.satisfice.com/kaner/?p=15
As I think about schools today, there are two ways in which I think I differ with you, James:
(1) I think that the division into schools can be an extremely useful analytical structure without being a religion (for example, I can look back at the great behaviorist versus gestaltist fights of mid-20th-century experimental psychology, gain a much better understanding of the field by viewing competing research at that time through these different lenses, but ultimately ascribe to neither school myself.).
(2) I think that it is inappropriate to place individuals in schools that they don’t identify with. I think it is fine to say that a specific paper by JoeX illustrates the kind of thinking that we call Factory School, but I don’t think it is appropriate to call JoeX a Factory Schooler unless he would call himself that.
There are foundational differences in our field, in our use of terminology and in the patterns we look to first as guides for practice. The schools approach is one way to organize and study the clustering of differences, into what look like different world views. As such, it can be of great value even if many people in the field refuse to identify themselves to any of the schools.
I think we grow impatient with some of the rejections of the idea of schools or the value of schools because we keeping running into some specific people who have a strong business interest in pretending that there is no controversy in the field. Unlike university studies, which (should) teach students to revel in controversy and critical analysis, the IT certifications (including testing certifications) generally present a one-true-way view of the field that is easy to teach, easy to examine, easy to certify against, and in my view, mind-numbing and therefore unfit for use in testing (which, surely, is applied critical thinking). Some people do business in the IT certification marketplace because they genuinely believe in the my-size-fits-all view of the world. I disagree with these people, might not want to hire them for anything, but don’t challenge their ethics. Other people in the same marketplace know better but promote the oversimplification because it makes it easier to advertise and sell their courses and their pet certification. It is easy to generalize, from the (I think correct) perception that this second group are charlatans to a broader impatience with others who just don’t find the idea useful.
Reading Cem’s reply and the way whole school discussion, I have following observations to make. Considering that I have been a die-hard supporter of school concept and argued in favour of it in all possible forums and conferences. I grew very impatient when people rejected the idea of schools, I felt like banging my head to wall and got frustrated. It bothered me that majority of the testing world (“general public”) are not likely accept the concept in near future.
1. While school concept for many (people who subscribe to Context driven school) is logical and useful and some others (regardless of their knowledge and intention to (or not to)learn sapient testing) is divisive and useless idea. I like the idea I use to communicate, others do not … so be it. I probably should not be losing my sleep over that.
2. There are factors like individual philosophy, business and career interests, personal agenda and others that make people to reject the notion of schools just like the personal philosophy (it suites me, I agree with it fundamentally, my testing is like that …. etc) and other personal interests cause people to accept the same concept.
3. Inspite of relentless attempts to drive home the point about schools, I have not been successful to evangenlize the school concept to people who simply say ” this does not work for me, I dont want to be labelled or I can be a good tester in my whole life without knowing about schools at all”.
I must say, I am tending to put a logical stop to my chase or attempt to take school concept to center stage… simply because factors other than pure logical/analytical/technical considerations driven people identifying themselves to schools. This is way too big space to cater to.
As I see many distancing themselves from schools concept or (literally) disowning the affiliation to school, can we say the time has come to re-invent the articulation of entire school concept?
Shrini
[James’ Reply: Anyone is free to reinvent anything. If I like it better, I may use it, too. But the schools concept is well established in other fields, and frankly I see it as infantile and uneducated to reject the idea of schools in software testing. If I’m the only one who talks about schools, so be it. It will still serve a purpose for me: to discuss the very real philosophical divisions in our craft.
People who refuse to acknowledge the concept of schools, and have no alternative way of acknowledging our divisions, are simply choosing to live in Neverland. They wish not to grow up.]