The Context-Driven School of software testing is a way of thinking about testing, AND a small but world-wide community of like-minded testers. There are other, larger, schools of testing thought. But CDT represents my paradigm of testing. By paradigm, I mean an organizing worldview, an ontology, a set of fundamental beliefs. When I say CDT is my paradigm, I mean that I believe all rational, educated people should want to understand testing in terms of the precepts of the Context-Driven School. This is like saying that I believe every rational, educated person should believe in microbes, atoms, genetics, and that the Sun does not revolve around the Earth. If you don’t believe those things, I can still like you, I just won’t hire you as a scientist.
CDT is not a style of testing. It’s not a toolbox of methods. It’s more fundamental than that. You could think of CDT partly as an ethical position about testing. All methods or styles are available to Context-Driven people, but our selection of methods and reactions to testing situations are conditioned by our ethical position. This position is defined here. It’s also exemplified in the book Lessons Learned in Software Testing.
It’s a Way of Thinking AND a Community
CDT can be confusing for several reasons. One is that it has a dual nature. It’s a way of thinking as well as a community of people. Let me try to sort that out.
Anyone who thinks and works by the precepts of the Context-Driven School is a member of the school, whether or not they know that or acknowledge it. And if they think and work by principles that are similar, then they are a member of some other similar school that may or may not have a name. And if they think and work by principles that are hostile to CDT, then they are a member of some other school of thought, entirely. A “school of thought” is a pattern of thinking that at least one person engages in.
If you don’t know what you think about testing– if you have no beliefs, habits, understanding, etc.– then I guess you are not part of any school of thought. Otherwise, you are. Don’t whinge about it.
HOWEVER, what if you aren’t a “joiner” and you don’t want to be seen as “belonging” to any particular testing community. There are a lot of testers like that. Okay then, you can say that you’re not a member of any school of testing. You can even say that the whole idea of schools bothers you. If that’s your feeling, then I bet what’s happening is that you are reacting to CDT as a community, rather than as a worldview. But in the end, unless you just have no ideas and beliefs at all, you don’t have the option of living without a way of thinking. You only have the option of not describing and labeling that way of thinking, or not associating with people who think of testing in the same way as you do.
Why Have “Schools of Testing?”
It saves a lot of time and energy to acknowledge that there are basic differences and controversies in the craft of testing. We can acknowledge that and move on.
Here’s an example: A particular woman wrote a book with the word “testing” in the title. But her work does not bear upon my work. She uses assumptions and definition of words that I reject. In a very real and practical sense, I don’t actually consider her to be a tester, although she would say that she is. Given that, I don’t bother to attack her ideas. I don’t bother to find common ground. There is no interesting common ground between dissimilar paradigms of testing. I imagine she feels the same way toward me.
She has a perfect right to publish and speak at testing conferences. But I’m going to ignore her, pretty much. My shorthand for that is to say that she belongs to a different school of testing. I might characterize her school in general terms, sure. I might say she’s from the Agile School or the Factory School or the Quality Control School (in fact there are three different women I’m thinking of, each of whom has written a book with testing in the title, whom I would describe that way). That’s about all I’m going to do.
I save my energy by working within my school to deepen the testing craft as I see it. I comment on other schools rarely, and when I do the motivation is usually to prevent my readers and clients from thinking that there is only one school of testing thought. This is not the same thing as saying that all of us inside a school have to agree. In fact, one of the basic values of CDT is that we argue with each other. We challenge each other. We do have difference styles of testing and different opinions about practices.
A second reason I like the schools concept is that it helps me quickly establish my reputation with someone new. “I’m a context-driven tester. Go and Google that.”
The Basic Idea of Context Driven Testing
Here is the basic idea: Because testing (and any engineering activity) is a solution to a very difficult problem, it must be tailored to the context of the project, and therefore testing is a human activity that requires a great deal of skill to do well. That’s why we must study it seriously. We must practice our craft. Context-driven testers strive to become the Jedi knights of testing.
Aside from the idea, this is also a community. It is a world-wide movement. The most prominent leaders within the CDT school include: Cem Kaner, James Bach, Jon Bach, Michael Bolton, Doug Hoffman, Paul Holland, Matt Heusser, Mike Kelly, Rob Sabourin, Ben Simo, Henrik Andersson, Ajay Balamurugadas, Shrini Kulkarni, Pradeep Soundararajan, Bernie Berger, Selena Delesie, Sajjadul Hakim, Julian Harty, Karen Johnson, Jonathan Kohl, Tobbe Ryber, Meeta Prakash, S. Dhanasekar, and Jerry Weinberg. I may have left a few people out. This list is off the top of my head.
(At the time we declared ourselves, Jerry Weinberg told me he’s not a member of the school. Except that, philosophically, I think he really is. Just by being himself and doing his own thing, Jerry’s work has been a beacon for the context-driven crowd. I consider what I do to be an ongoing reference to the life work of two people: Jerry Weinberg and the late Herbert Simon. I try to honor them by being a good tester.)
It’s not a Priesthood
Context-Driven Testing is an open community in the sense that anyone can speak up and contribute. The way you become leader in our circle is by having ideas, offering them publicly, and engaging in debate about them. Then, as your ideas are tested, you earn your reputation. I just met a fellow named Vernon Richards, for instance. He spoke up during the Nottingham Testers Gathering, and I was impressed with his questions, comments, and general attitude. He spoke with warmth and conviction. See? I didn’t even know who he was, and now I’m talking about him. That’s what I mean by reputation. It’s a start. I’d like to see him share more of his experiences at conferences.
The CAST 2011 Conference is All About Context-Driven Testing
If you look carefully at the innovations in testing practice and thought, I bet you’ll notice that most of them are coming from the Context-Driven community. And the CAST 2011 conference is the first conference devoted to the ideas. My next post will be about that.
“When I say CDT is my paradigm, I mean that I believe all rational, educated people should want to understand testing in terms of the precepts of the Context-Driven School.”
As Jerry taught me through his books, humans are unfortunately not rational, though hey like to pretend to be that. But rather humans act and think upon emotions.
[James’ Reply: Humans behave in both rational and irrational ways. Rationality is like singing: You don’t automatically do it when you make noise, but everyone CAN do it, many people do it badly, and anyone can train to do it better. What I am talking about are people who are trying to behave rationally.
BTW, a nice definition of rationality I picked up from the book Thinking and Deciding, by Jonathan Baron, is that rational thinking is “the kind of thinking that helps us achieve our goals.” This reminds me of one definition of science: “science is what works.”]
You might consider this reply to be an emotional reaction to the sentence from your blog post, for example, but similarly you can consider whether or not someone wants to be part of the context-driven school – in fact any of the Schools, as you once told me, in the Kuhnian sense – you could consider any decision to be an emotional reaction.
[James’ Reply: Decisions involve a lot of emotions. For me, one powerful feeling is that I want to be extremely good at what I do. Since what I do is find important problems in software, I need to be Context-Driven.]
Reaction to what? Reaction to what you wrote maybe in this blog post. That said, I consider myself a context-driven tester, but still seek to understand when and why someone else don’t want to be part of a school.
You mentioned the quality control school and the agile school of testing. However, I would wager that most of the people associated with these schools do not label themselves with such names. Because of that, it is hard to find any information on them. Could you please describe your idea of these schools in more detail?
[James’ Reply: Sure. I see five schools: Agile, Quality Control, Factory, Analytical, and Context-Driven.
Agile (the only other school that names itself) sees testing as an adjunct to programming, preferably automated. They accept that exploratory testing is a good thing, but don’t feel that there are any special testing skills that deserve or require development.
Quality Control sees testing as an admission of process failure. Someone in the Quality Control school of testing keeps reminding everyone that bugs must be prevented, rather than found. They are testers but don’t want to be testers.
Analytical (mainly academics) see testing as a fascinating mathematical problem that they are content to solve in well-bounded research exercises, and then hope to scale up.
Factory (the biggest school) sees testing as a matter of systematically manufacturing testing artifacts. They focus on documentation, metrics, and detailed instructions intended to control tester behavior. They love automation. T-Map and TPI are great examples of the factory school in action.]
I would count myself as part of the Agile testing school of thought, but I practice context driven testing within an agile methodology. I feel that exploratory testing is a skill that deserves and requires development. I also believe in a “whole team approach” where testing is not just about uncovering bugs but preventing them, does that make me fall into the Quality Control group as well?
[James’ Reply: This makes you confused about what you believe, I think. This is not a problem, because you’re still young. I don’t know what school of thought you are a part of, but right now you are hanging out with people in the Context-Driven community.]
On some level cost plays a role and if low severity, high exposure bugs can be prevented through process and collaboration I am able to focus on honing my ET skills further by uncovering more complex issues. Finding bugs or “B-time” to borrow a term from SBTM, leads to clerical overhead. It causes time to be taken away from the valuable art of testing. If I can eliminate these bugs by working with the product team and development, am I not setting the stage for richer exploratory testing sessions? I suppose one could argue that the “Whole team approach” trades “B-time” for “S-time” since attending requirements meetings and collaborating with product owners is also not exploratory testing, but I would argue the added value is that with “B-time” you’re just catching raindrops, whereas with the “S-time” involved in a “whole team approach” you’re helping to teach your craft to other leaders within the organization. Maybe implanting a thought in the backs of their minds. “What would our tester think of this?”
I am discovering as I write this comment that there may be several schools of testing, but they can, and often do, intersect and the best approach for me is to identify value in these schools and adopt those precepts where they suit my purpose.
I guess you could say I take a context-driven approach to schools of testing.
[James’ Reply: Yes, many people say that who don’t know what a school is. Do you also say that about religion? Are you a Jew sometimes and a Christian other times, and then sometimes a Zoroastrian? Like religion, the school you belong to is not a tool you pick up or put down, it’s a commitment to a set of beliefs. If you are not committed, that’s cool. You’re still seeking.]
James said, “Agile (the only other school that names itself) sees testing as an adjunct to programming, preferably automated. They accept that exploratory testing is a good thing, but don’t feel that there are any special testing skills that deserve or require development.”
I consider myself an Agile Tester and a member of the Agile community. There are a bunch of us (including Markus Gärtner who commented above) in the Agile community who tend to focus on testing-related topics. I suppose you could say we’re an Agile Testing community.
But I don’t see “Agile Testing” as a school in and of itself. Rather, Agile Testing is what you do in an Agile context. Thus, we’re context-driven.
[James’ Reply: I have a long experience bumping up against Agilists, including you, and I would say you definitely do have a different way of approaching and thinking about testing than I do. If you recall, this has led to bad feelings between us, at times. As you know, I have very different beliefs about what constitutes exemplary professional behavior than you do. It’s not just a different style. It’s not an accident of context. It’s a different set of values, a different ontology, a different vocabulary. We study different things. We value different skills.
The context-driven school is not simply about understanding context, it’s about avoiding pre-judging the context, and about developing a testing skill that can operate in a wide variety of contexts. It’s also about humanism (this is how, early on, the Context-Driven and Agile people connected). I don’t have a preference for projects that use “Agile” methods. I just study how to test things really really well, and then I apply that to whatever project I’m working on.]
It’s just that many of us, including me, choose not to work in non-Agile contexts. Working with teams that embrace Agile practices and values is just so much better than working in traditional environments where people work in silos and where shipping software involves a lot of late-cycle drama.
Even if we called Agile Testing a “school,” I disagree with the way you characterized it.
[James’ Reply: I characterize it the way I see it, as I visit companies that practice Agile and non-Agile approaches around the globe. You have a lot of experience… So do I. Anyway, offer an alternative view, if you want.]
To say that Agile Testers, “accept that exploratory testing is a good thing” downplays the role of exploratory testing in Agile. As far as I am concerned a story isn’t Done until it’s been implemented, checked, *and* explored. Exploratory testing is thus a mandatory part of Agile, not just a “good thing.”
[James’ Reply: That’s nice, but I would be more impressed if I saw you guys doing any sort of studying of exploratory testing. I have consistently been disappointed at the shallow approach taken to the study of testing skills. Please point me to sources that will help to change my mind on that. My experience with Agilists is that they work on their programming skills, but simply assume that they have testing skills. When I complain about that, they pull out the old “no specialists” crap, which seems to be a blanket excuse for not learning anything other than programming.]
Further, to say that Agile teams “don’t feel that there are any special testing skills that deserve or require development” is false. Testers, with their understanding of how software tends to break, help the team prevent defects.
[James’ Reply: Elisabeth, I appreciate that you may be hanging out with different people than I encounter in my travels, but that does not invalidate 12 years of my experience with Agilists. You say it’s false. At best, that means there is at least some pocket of Agilists who study testing skills. So, does that mean you do? Can you point me to any signs or symptoms of your study of testing skills? I haven’t seen much of that, and I would like to. Many years ago you excused yourself from debates with me, ever since Cem and I challenged your assessment of a study of exploratory testing. Are you open to debating now? Because that’s how I come to understand what you do and don’t do, NOT by politely accepting your word for it. In CDT we model ourselves on the notion of a self-critical scientific community.
I believe you are a gifted tester, and you know I have said so repeatedly and publicly, but that’s not the same as making it a focus of your work to develop your testing skills, follow that study wherever it takes you, and to develop systematic ways of teaching those skills to others. If you or anyone in the Agile community is doing this, I would like to know about it. Instead the big innovations in Agile are all about tools or about copying some Japanese concept or other.]
And that, perhaps, is the fundamental difference that you’re observing about Agile Testers.
Agile teams put more effort into preventing problems than finding them. The core practices in Agile–ATDD, TDD, CI, Pairing, etc.–are all tactics for preventing defects or finding them at the moment they’re introduced. The tight collaboration, high degree of communication and coordination, high degree of alignment, and fast feedback on Agile teams all mean that testing on Agile projects looks different than testing on traditional test-last projects.
But that doesn’t mean Agile teams value testing, or testers, any less. If anything, Agile teams value testing, and rocking good testers, *more* than traditional teams. Testers are an integral part of the team and the process in an Agile context.
[James’ Reply: I would like to believe you. Now, point me at some evidence that you take testing seriously as a skill set. For instance, what are the skills of testing, in your way of thinking?]
You said “a small but world-wide community of like-minded testers.”
I wonder how long we can keep on saying that. it’s a cool phrase that gives an impression of an elite group, that has got an edge over others, but is it really an small group?
My perception is that every other tester that I met (here in Finland or in internet circles) is swearing to Context-Driven school and is using words like “passionate exploratory tester” in their biographies. I may be wrong, but I think we are dealing something that starts to be an “automatic” starting point for every tester instead of a distant goal that really is something to strive for
[James’ Reply: Really? I happy to hear that. And truly surprised.]
James said: “My experience with Agilists is that they work on their programming skills, but simply assume that they have testing skills. When I complain about that, they pull out the old “no specialists” crap, which seems to be a blanket excuse for not learning anything other than programming.”
My colleague Stefan Roock introduced Testing Dojos (http://testingdojo.org) to a client of ours. Testers and Programmers together meet up in order to see how Exploratory Testing can be done, and share their experiences. The company uses Scrum and Kanban in the large. Unfortunately this is an exception for Agile companies, still this is an outstanding example.
[James’ Reply: I reviewed the site. It’s a very nice start. What I notice, however, is the lack of specificity about testing skills. What is described is an opportunity to practice testing, but what comes out of it? I notice FCC CUTS VIDS is mentioned. That comes from Mike Kelly, who in turn adapted it from his one-on-one test coaching received from me when he visited my home in Virginia, years ago. That’s a specific pattern of testing to practice with, but I note that it comes out of the Context-Driven community. It’s an artifact of our attempt to study the craft.]
I’m confused,
I feel as though your comments on Elisabeth’s post directly contradict your comments on my post.
[James’ Reply: Okay, how so?]
I am an aspiring context-driven agile tester who puts a huge focus on testing ability. I don’t know that it’s something to brag about but all of my efforts towards developing tools and skill sets around programming/automation have been derailed so that I can focus on the “way of thinking” that seems to be the focus of CDT. I don’t pretend to have anywhere near the understanding of CDT that you do. However, I know that my job, first and foremost is to add value for my stakeholder and customer. I feel like to say “CDT” is religion is to say that there is only one way to approach it. I challenge the analogy, is there a fundamental conflict between the two schools as in Judaism and Christianity? Why can’t I practice Santaria? Maybe I believe in Catholicism and Voodoo? Does that mean I don’t get into the kingdom of heaven (ostensibly meaning I can’t be very good at what I do?)
[James’ Reply: Adam, it’s a worldview, a paradigm. It’s not a toolbox. It’s not a style or approach. I use the religion analogy because religion involves an exclusive cosmology, and so it’s a little easier for people to understand that you don’t pick up and drop paradigms like they were a suit of clothes. Yes, there is a fundamental conflict among the schools. If you try to argue with people from another school, you will be impressed with how far you don’t get.]
I am exceedingly grateful to you and the rest of the CDT community, and I’ll continue to extol the virtues of CDT. I am also an agile tester. I believe in a whole team approach. I believe it’s better than a traditional siloed structure for many reasons I won’t go into here. Does that mean I can’t learn from CDT and apply those skills to my testing? Do I have to choose one, or can I attend the Synagogue and Mass? Maybe I’m keen on aspects of Judaism, and want to combine them with aspects of Christianity to serve my belief. Perhaps the only thing I reject is exclusion by either group?
[James’ Reply: CDT is not against any approach, pretty much, so of course it’s not against the whole team approach, either. And no, you don’t have to choose one school, but of course that means you are living with a self-contradicting and generally inconsistent belief system wherein you will find it hard to gain respect in any of the communities in which you dabble. Certainly you can create a brand new school, too. But I doubt that’s what you are doing.]
I can say, with a degree of certainty, that I am in the job I have because of testing ability. In fact I told my (extremely agile focused) boss before he hired me that I had no automation skills and could not program. He said he was confident I could learn whatever was required and that first and foremost he cared about hiring somebody who was passionate about testing and self-education. If your statements about agile were broadly true, he would have hired somebody who had the skill with the tools, since testing ability are less important.
[James’ Reply: If my statements about agile testing are broadly true, then neither he nor you would have a strong idea of what testing skill is, nor how to develop it, nor have a focus on developing it. So, does he? Do you? Tell me how.]
Hi James,
Thanks for the post. I thought it was interesting and I’m looking forward to the follow-up. I had some comments on it:
“The Context-Driven School of software testing is a way of thinking about testing, AND a small but world-wide community of like-minded testers.”
Can you can separate the two? The “way of thinking” changes as new ideas are developed by the community, and the community changes based on new ideas in the “way of thinking”.
[James’ Reply: Yes you can separate the two. A community is a group of people who work and live together in some way. I can imagine two completely separate communities of people who both have the CDT mindset.]
“You could think of CDT partly as an ethical position about testing.”
By ethical, do you mean “I have the responsibility to perform the best I can in a given situation because it is the right thing to do”?
[James’ Reply: Not really, but close. By ethical, I mean it constrains behavior based on values. For instance, it involves treating testing as primarily a human process done by and for humans.]
Also, you say “PARTLY as an ethical position”. If it’s not just ethical, can you elaborate on the other part?
[James’ Reply: Sure. It’s not only an ethical position, but it’s an ontological and epistemic position, too. It claims there is no such thing as a best practice, for instance. That’s not an ethical statement, it’s a statement of fact within the way we look at the world.]
“HOWEVER, what if you aren’t a “joiner” and you don’t want to be seen as “belonging” to any particular testing community. There are a lot of testers like that. Okay then, you can say that you’re not a member of any school of testing.”
Can you define the difference between school and community? Can you be in the community, but not be in the school? What about the other way around?
[James’ Reply: My wife and son are my family. That’s a little community. A community is a group of people whose lives intertwine in some way. A school of thought, in this context, is a pattern of thinking that defines and guides how you see your world. Generally speaking, people who see the world similarly will form communities, but they may not. Personal animosities, for instance, may prevent that.]
“But I’m going to ignore her, pretty much. My shorthand for that is to say that she belongs to a different school of testing. “
So are you saying that no two schools can get along?
[James’ Reply: I’m saying they don’t agree on the meanings of words or on what matters and what doesn’t matter. Otherwise, I guess they can get along fine!]
As a tester who believes in context, shouldn’t I review all possible schools of thought to determine the best course of testing action for my client given the context?
[James’ Reply: No, because you don’t choose your school of thought based on transient context, any more than you would convert to a new religion based on which one has a nicer holiday coming up.]
Different schools of thought reminds me of the frustration I have with politics. Both parties have the same basic job to do, and in my opinion, both parties have some good ideas. Wouldn’t the best governing party be one that takes the best ideas from all schools of thought and puts them to use?
[James’ Reply: Nice idea, but of course that’s impossible. Not only does it empirically not work (just look around you!) but there is no ultimate perspective from which one could choose a “best idea.” Look up “perspectivism” for more on that. There are no privileged perspectives.]
“testing is a human activity that requires a great deal of skill to do well”
I couldn’t agree more with this. Too often people want to take the human element out of testing as soon as possible.
“The CAST 2011 Conference is All About Context-Driven Testing”
I’m relatively new to the testing community, but for some reason I always associated AST with context-driven testing. so when I saw this year’s conference was on context-driven testing, I thought “Yeah, isn’t it always?”. I guess the answer is “Not always”. I bought my ticket to CAST yesterday and signed up for your tutorial. I’m looking forward to being challenged and learning a lot.
I struggled a bit when I originally decided to reply to this blog post. I was finding it difficult to collect my thoughts in a way I liked. I finally came up with a method that worked for me, which reminded me of one of your Buccaneer great secrets “I can learn on purpose while also creating opportunities to learn by accident. I do both!”. The learning on purpose is the reply I’ll get from you based on my comments. The learning by accident is how I figured out an effective way to collect my thoughts when commenting on a blog post.
Thanks again for the interesting post.
[james’ Reply: Thanks for replying. This is helping me create my tutorial.]
I had just finished an article on Context Driven Testing, and thought I understood it, but I guess not…
In my article I was comparing ISTQB, TMap and Context Driven testing as three different approaches to testing. But if I understand you correctly, CDT is on a higher level so to speak (a higher level of abstraction perhaps)? both ISTQB and TMap can be grouped in the factory school and not only TMap I think? Both TMap (Next) and ISTQB also give a lot of attention to prevention so can I label these approaches as both Quality Control and Factory?
[James’ Reply: ISTQB is a sort of community, although I think it’s not a testing community, but more a CV preening society. When I talk with ISTQB-oriented people, it’s like talking to parrots who randomly quote bits of books they read– or didn’t read. It’s a community that believes in authority-based reasoning, which is anathema to me.
Anyway, you can categorize how you want. What I look at is fundamental beliefs. The quality control school is identifiable by how they keep saying that testing is beside the point. They don’t want to test, but are willing to while waiting for the great day when the Great Prevention Savior comes.
Factory schoolers believe in an activity called testing. They think of themselves as professional and organized. They want to squeeze out the human element of testing because it’s icky and mushy and can’t be measured or controlled. That’s the impression I get from TMap, etc. I also think that’s a losing enterprise, but it’s intuitively appealing to people who lack social skills or have a great social skill for manipulating other people, or just don’t know much about testing.
I look at the TMap stuff and wonder how the author of it ever convinced himself that he knew how to test– because there’s nothing about testing in there! Just documents, documents, documents…]
What I’m trying to say is: schools are mutually exclusive, but can (on the surface at least) have things in common? To take the example of Religion: every christian church has a view on Sacraments. From the outside they look the same, but actually they are very different things.
Thanks a lot for this article, this really helps a lot to get a better picture of what CDT is as opposed to what is does.
[James’ Reply: What you must understand is that “things in common” is a meaningless idea when crossing paradigmatic boundaries. When I say “test” I’m not talking about the same idea as the Factory schoolers. For me, test is a verb, for them, I have the impression it’s a noun. For me, testing is an activity that testers do, and if you want to understand it you must understand testers. For them, testing is a collection of “test cases” which have some sort of structure and content that they believe they can measure and control. So, we use the word “test”, yes, but that is not common ground.
What I’m saying is obvious old hat stuff to anyone who seriously studies the history and dynamics of human ideas. Psychologists, anthropologists, semioticians, ethnographers, cognitive scientists, etc. all know how treacherous it is to declare common ground. But these are not subjects that people outside the Context-Driven School like to study.]
James,
You often talked about religion as metaphor for school of testing. How do you then describe an “athiest”? Someone who rejects the existence of ANY “fundamental” belief about testing? How do you describe people who practice many religions at the any given point of time (In india we do have such instances and is accepted social norm – there are cross religion marriages where after the marrage both partners continue to practice their original religions and also practice each other religions). In many comments here – people are asking can mix Christinity and Judiasm? Can I take best of both religions?
[James’ Reply: I use religion as an example so that people will be more cautious. But if you want to really understand the underlying idea then don’t bother to think about religion, instead, study the history of human thought in general. Read Thomas Kuhn, Imre Lakatos, Paul K. Feyerabend, C.S. Pierce, Levi-Strauss. Study the problem of translating text from one language to another.
To answer your question, no one practices many religions. The appearance of practicing many religions comes from adopting the surface rituals of different religions, or from the fact that the different religions are actually not different at all.
You can’t take the “best” of both religions, because there is no independent perspective from which “best” can be determined. All you would be doing is taking your favorite things from each religion, which is not the same as having multiple religions. Instead, that’s creating a new religion.
If your new religion is reasonably satisfying and self-consistent, we might call it a school of thought. If it is not, then we just say you’re feeling confused. However, it is not possible to add disparate elements from different schools together without fundamentally warping and transforming their basic meaning.]
Extending on what Adam Yuret mentioned – Can there be an Agile Factory Tester? or Agile CDT Tester? My answer is NO. Following Kuhnian incommensurability of paradigm – people in two paradigms (now I am using paradigm and school interchangeably) live in two different worlds. I think, the Fundamental beliefs and world views are mutually exclusive, thus you can not have mix of one or more schools. Right?
[James’ Reply: Certainly you can create a new school that combines what you see as elements of other schools. But in doing so, you change those elements. You don’t merely import them.]
>>>> By ethical, I mean it constrains behavior based on values. For instance, it involves treating testing as primarily a human process done by and for humans
So what you are saying here is “treating testing primarily a human process done by and for humans” is a ethical position congurent to values of CDT. Someone with Factory testing school might say “Software testing is a repeatable factory process done in a controlled engineering environment is with precise requirement and results” as her ethrical position congruent to the values of her school?
[James’ Reply: Yes.]
>>> A community is a group of people whose lives intertwine in some way. A school of thought, in this context, is a pattern of thinking that defines and guides how you see your world. Generally speaking, people who see the world similarly will form communities, but they may not. Personal animosities, for instance, may prevent that
You seem to be making distinction between “school” and community in a sense that one (school) is abstract or pattern of tihnking a world view where as community is real – made of up of people signing up for the school. Right?
[James’ reply: Yes.]
My response to the question “Do I need to choose any school” is – “choice” is wrong word in a sense is that when you are initiated into testing – either by your professional needs, influence of some one in the organization or community or ones own practice – you start forming a world view through which you start seeing the world of testing. It involves a subtle selection. I suspect many who reject the notion of schools – have an inbuilt world view associated with them which constantly creates, shapes and conditions everything they do with respect to “testing”. You can compare people getting into tesitng to kid learning things (walk, speak, cry, shout) – we don’t say kid is choosing things in behavior but we say kid is learning.
Like kids learning about their new world (hence forming their world views) – people getitng initiated into testing will form the world view that is we call as “school of testing”.
[James’ Reply: Yes, I agree.]
>>> Yes, there is a fundamental conflict among the schools.
Do you see a possiblity of the fundamental conflicts being resolved ever in future? What is the root for these conflicts? Human Nature? diversity in thinking, culture, personal motivations and social influences?
[James’ Reply: No, it doesn’t get resolved. What happens is that adherents of an unsatisfying school abandon it (perhaps by dying). Galenic medicine, for instance, has been abandoned, but nothing was resolved.]
>>> It’s a different set of values, a different ontology, a different vocabulary. We study different things. We value different skills.
Right. That is what exactly notion of schools stands for – when we say two people belong to different schools – it means these two people hold two diffrent (often mutually exclusive and unique) world views – study different things, value different skills.
[James’ Reply: Yes, however we may use the same words, and so fool ourselves into thinking that we agree.]
>>> Elisabeth : Agile teams put more effort into preventing problems than finding them. The core practices in Agile–ATDD, TDD, CI, Pairing, etc.–are all tactics for preventing defects or finding them at the moment they’re introduced. The tight collaboration, high degree of communication and coordination, high degree of alignment, and fast feedback on Agile teams all mean that testing on Agile projects looks different than testing on traditional test-last projects.
Notice how focus of agile testing or agile testers shifts from testing (investigation to discover bugs) to programmer centric (mostly) work of “preventing bugs”. In CDT – I have not seen many talking mainly about “preventing bugs” – we talk more about enhancing our cognitive skills focussed on finding hard bugs/problems. Is a distinct pattern of our CDT thinking? or CDT folks are often called or work in context where finding problems is the focus than preventing them? Then there are these “test first” or “test last” approaches – we in CDT seem to be saying “Test all the time or test as early as you possibly could and go on till you are asked to stop”.
[James’ Reply: Yes. I’m a tester. I love testing. I’m not interested in preventing problems. (I’m not saying it’s bad to do that, just that it’s not my business.) I don’t care what people do to prevent problems, any more than I care if a new computer language is invented. So what? Whatever they do for prevent problems, there is still the question “Is there a problem here?” I study how to answer that question.]
>>> Elisabeth : But that doesn’t mean Agile teams value testing, or testers, any less. If anything, Agile teams value testing, and rocking good testers, *more* than traditional teams. Testers are an integral part of the team and the process in an Agile context.
Notice also – how fundamentally this is diferent from CDT’s view about “Testing” – We don’t ever go around and say “testing is integral part, testing is valuable and so on”. Why? because it is “testing” that all we do nothing else. Instead of saying testing is part (integral or separate, first or last) of some bigger thing — we ONLY talking about testing and skills about doing good testing in ANY context. I can’t think of myself and someone from agile testing talking testing in same way. To repeat our values are differnt and we focus on different skills.
[James’ Reply: Yes. Good point.]
Lastly – the question that Markus asked – “what is the motivation (or trigger) of you to write this post” ? CAST? Your tutorial? or something else
[James’ Reply: I started writing about CAST and it turned into a post about schools.]
Asking few unanswered questions and some followup questions:
1. Continuing on religion metaphor – How do you describe an atheist in schools of testing sense? someone who rejects existence of any schools?
[James’ Reply: No, of course not. An atheist has an alternative cosmology that doesn’t make reference to supernatural beings or forces. An apatheist (like me) is someone who doesn’t give a damn about gods, regardless of their existence, in the same way I imagine bacteria don’t give a damn about the existence of elephants.
You can have a coherent world view or not. But if you don’t, then to that extent, you won’t have coherent opinions about the world. You can’t have more than one coherent world view simultaneously, since that means one or more of the following:
– The resulting synthesis would not be coherent (because you would have contradictory explanations for things).
– The different world views are not actually different (which is how they could go together).
– They are not world views (since there is only one world).]
2. Are values of schools mutually exclusive? you seem to agree that I can make a school that draws some ideals from one school and another set from another school and make a third set. Well… it might be that third is not necessarily the mixture but a totally a new school. But you would agree that such amalgamation is possible? but is such school self “inconsistent”? incongruent to some contradictions? (you would say then probably – one can work out the contradictions – as you can change the values)
[James’ Reply: Yes, they are mutually exclusive. I don’t agree that you can take things from different schools and put them together without fundamentally changing them. If you take the idea of castling from Chess, you can’t bring that into a game of Checkers– unless you change the meaning of castling, or change the meaning of Checkers. What you end up with is a new game. Only in a superficial sense is the new game a blend of the old games.]
3. When someone says “why not have only one school – let us have something made from selected ideals from many schools” – they are talking about mixing schools – but in a wrong sense. When you take the ideals of one schools and put it along with the one from the other… you can changing the basic structure of those ideals originally.
[James’ Reply: Yes. You have to ask “what is the basis of the selection?” Such selection cannot be rooted in either school, so where is it rooted? From what third and coherent position is this supposedly being analyzed?]
I see no conflict between the Agile Manifesto and Context Driven Testing. It’s ideas far more specific than the manifesto that cause a conflict, and I don’t think that is important.
[James’ Reply: I am a signatory on the Agile Manifesto. I also don’t see any conflict there. But that’s not what I’m talking about. I’m talking about attitudes about testing. I can be on an agile project, no problem. But I will be on it with my context-driven testing mentality.]
You know how religion is a set of ideals you adhere to, but perfection isn’t the goal. It is being true to the overall most important principles and becoming closer to your ideas that matters. I’m pretty much following that same process with my testing expertise. I know what is working in practice and what is unsustainable. I’m Context Driven because that works for those of us who are doing the actual testing. Look at the evidence. Try it. Improve. Demonstrate.
It doesn’t take too long if you work next to someone to see what school they are from in practice.
I think the revised story of the person who used the test tool is going to be more context driven than I intended. The client shipped the item under test without fixing the blocking issues that would allow me to even harness the power of the tool I set up to work around their schedule. Hey. Ok. The bonus is, there are other clients who might have other needs. Now I can try again. I’ll have something pretty real to say about context. Sometimes it changes way too quickly for our grandiose plans to be worth much.
You coined context-driven school to focus on testing skills in your comments. If there was one thing to mention before reading this, I would have mentioned the mantra “no best practices, only good practices for context” and come out with quite different understanding of context-driven. I think Cem Kaner had a post on context-driven / context-aware – apparently much longer ago than I realized, having followed these blogs for a while: https://www.satisfice.com/kaner/?p=45. I read these side by side and felt context-driven may be evolving and reforming.
Cem’s text always has been a key idea for me: “Ultimately, context-driven testing is about doing the best we can with what we get.”. In addition to skills you need for that (or as a skills you’d need for that), I feel need to recognize what we have and where we should get, and what part of that is “testing”.
[James’ Reply: Context-Driven is obviously about being context-driven. But that’s not in any danger of being neglected. What’s not so obvious is that in order to be context-driven, you must have the requisite skills to read and respond to the context. That’s why the key idea, for me, is actually about skill: skill in talking about testing, analyzing testing situations, and crafting solutions.]
Since I’m both a cognitive scientist and a tester I feel like I have found my “home” within the Context-Driven-Testing community. I can relate to pretty much everything you write. Thank you for keeping testing within the social sciences where it belongs. It’s also quite funny to read between the lines how you sometimes struggle to describe “it” in the best way possible, which indeed is a difficult task since “it” only manifest itself in context, ready to be manhandled by a Context-Driven-Tester 🙂
Not sure if the schools concept has helped testing or hurt it…..
[James’ Reply: I don’t understand what you are saying. The “schools concept” is a recognition that we don’t all think in compatible ways. Are you trying to say that it’s bad for us humans not to think in compatible ways, or are you saying that it’s bad for people like me to highlight that and stop trying to argue with people who won’t listen to me? Or are you commenting on my particular analysis of which people think in which ways?]
Contrast this with agile. The software development community has uniformly endorsed agile (for the most part). What I find most appealing about the movement is:
1. acknowledgement that excellence in engineering is indispensable
2. universal acceptance that we want to do what is best for customers. We want to prevent defects. Nothing else matters.
[James’ Reply: It is certainly not true that the software development community has endorsed agile, uniformly or otherwise! There definitely is widespread co-opting of the Agile concepts, yes. There is also widespread co-opting of the concept of ET, but that also is generally misunderstood.
Any movement based on skill and personal responsibility is doomed to be a niche movement, when the opportunity for incompetence to be rewarded is available.]
What is sad:
– there are many questions about agile. I don’t think they are game killers.
– the traditional approach actually worked well in some (many?) cases.
[James’ Reply: Not only did it work, but it still is working.]
Compare this with the context driven school:
– the context driven school has done some deep thinking over decades and you can find very little fault with any of their thinking.
– I think in the traditional model, testing did not work. There is a crisis in testing. Testing really needed change, even in the traditional model.
Testing *really* needed this change. Alas…
I find most testing done is shallow. What is disappointing is that this mass of mediocrity may overshadow something great – the context driven school.
[James’ Reply: I try to be a beacon for testers, and some have heeded that beacon. Most do not. I don’t see this as terrible problem. Human frailty is always going to be ascendant. We shouldn’t waste our time bemoaning that. Simply create greatness wherever you are, help the people who want help, and don’t be too attached to the outcome. This is advice I first apprehended from the words of Krishna, when I read the Baghavad Gita. It helped me focus my energy better.]
I think the Analytical, Factory and Quality Control schools should become obsolete. I think the context driven school should provide a vision for how it would do testing in an agile environment (not being constrained by agile). I think that vision would encompass the agile testing school (with modifications).
[James’ Reply: I have provided that vision in various places and ways throughout the past decade. This is what I teach. It’s called Rapid Testing.]
I think the context driven school should provide a vision on how testing would be done in an IT environment, with concepts like UAT, etc.
[James’ Reply: Yes… Done.]
I think testing needs a movement like agile. The context driven school is such a movement.
I think the schools concept helps “keeps the peace”. However, having dysfunctional schools will not help in the long run. I don’t blame the context driven school for this. I think it’s a failure of the test community at large.
Just being honest…..
[James’ Reply: I don’t control who thinks what. The Quality Control school of testing is essentially self-destructive (since they insist that they would rather not be testers), and when they are successful (at bug prevention) they do help software be better, so I’m not worried about them. The Analytical school is generally harmless, sometimes helpful, and they are confined mostly to universities. The Factory school is the dangerous one, but also, their way of thinking does not work well, so perhaps market forces will eventually make it irrelevant. The Context-Driven school exists to solve the problems that motivate testing. Ultimately, one would think that solving problems skillfully should be a popular model, except that it’s hard to attain the skill necessary to do that. So, I expect we’ll see the Factory school persisting for another 50 years or so.]
James,
Bret Pettichord also wrote about testing schools. He uses a different name (standard school instead of Factory school), but otherwise I believe it to be the same categorization. Am I correct in this?
[James’ Reply: It was Cem’s idea to name our school, Bret’s idea to name the other schools, and my idea to call the Factory school “factory.” Bret felt that was a pejorative term, but of course it isn’t. Factories enable mass-production. If you really think testing is should be a form of mass production, then the factory model would be a great model for you. “Standard” misses the point. The factory people aren’t interested in standards for their own sake, they are interested in them because they enable mass production, automation, and they help eliminate the troublesome human element.]
Hi
If they want to successfully eliminate the “Troublesome human element” from testing then they need to eliminate it from development also. Otherwise this happens : http://kristjanuba.wordpress.com/2011/03/02/to-merge-or-not-to-merge/
And the best part of Context-Driven Schools is that the ideas are out there to be challenged by anyone. Not made by some ‘elite’ persons that give out the “Thou-Shall-Follow-These-Steps” books and exams.
I like the term “tester”. It provides a common name for a group with similar interests, the common name helps unify the group, so we can learn from each other. Engineer, pilot, printer, welder, geologist, knitter, writer, etc. all will have different approaches that can add to that body of knowledge.
If two welders meet, the name welder provides a common ground to communicate about welding so they can learn from each other. Having two segregated camps “the Tig school of welding” and “the Mig school of welding” to me would seem to add more of a barrier to communication and to the advancement of the craft.
We who call ourselves testers, all practice the craft daily, and we “all” have very complex human brains. Whether we are aware of it or not we all have something to add to the body of knowledge of testing software, and just by testing and communicating to each other about testing we add to it. The same way the hand axe became the computer over time.
[James’ Reply: I like the term tester, too. But it means something different to me than it does to people in other schools. That’s why, after a good ten years of arguing with people in public about what “tester” and “testing” means, I thought it was a good idea focus on developing the vision of testing shared by the people of my school.
You seem to have reversed the causality of the schools idea. We haven’t declared schools because we think that’s better than not having schools. We have declared schools because schools already exist. If, for instance, TIG welders thought about welding in a fundamentally different way that MIG welders (not merely having a different style of welding, but having a different definition of welding), then declaring different schools of welding would not be creating a division– the division would already be there.
If you want an example outside of testing, look at the field of Statistics. The Bayesian school and the Frequentist school of statistics are both productive and both provide an account of statistics. They even agree on some of the mathematics. But what they think they are doing when they do statistics is profoundly (not just incidentally) different.]
There is a link between context and schools of thinking and I find it hard to describe.
The best way I can describe it is, that where you work, the other people you work with also have “schools of thinking” of their own. If most people in a company share a school (e.g. on how to develop or test software) then that is an important part of the context you work in. I suspect this can have a huge impact, because if you do a great job according to your own school but not according to others (e.g. your boss) I don’t think you will get very far. Assuming you want to be successful at what you do (and assuming ‘success’ is partly determined by your career/ how much your employer appreciates your work- which it may not) you need to find a way to make your own school of thinking compatible with your context (environment/ testing community/ employer).
I haven’t really figured out to what extent you can (or can’t) adapt your school/ paradigms to be compatible to a (changing!) environment just yet.
Francien,
I may be wrong (i’m relativly new to testing on the whole) but i think a large part of what CDT gives you is to not be bound to the process that surrounds you. If today the project is agile, great, i have the knowlege and skills to work with this and i know x,y and z will allow me to do a good job testing here. Tommorow the project is a test-last project, that’s ok too. Again i have taken the time to study testing and how to be a good tester so i can use these skills to do a good job.
The company i work for currently is very much based around the factory school when testing. Becasue of this I have to write test cases. So i use these test cases as markers of what i am actually testing. I can run the test and see how the piece works, from this i can see potential risks and things i can try to exploit to find bugs. it provides a focus point for me, not defines all of the testing i carry out.
In my opinion being in an enviroment you are asked to script tests does not me you are confined to the limits of those scripts, if you go beyond the script and explore people will see the results as you out perform the factory testers day after day.
equally if you are in an agile enviroment, as a context driven tester this is just the frame within which you do your testing and what allows you to make choices of how to test the product well.