I recently had the opportunity to converse about tester certification with Carol Cornelius, who’s on the board of the New Zealand version of the ISTQB. The discussion went well in one respect: she did not physically run away. (Oh, and she concurred with me on the subject of Stuart Reid, which was nice to hear.)
[Considering that they misrepresent my ideas, perhaps she should have run away. To give two examples, my work on SBTM has been plagiarized in their Test Management Syllabus (page 47 of the Advanced Level Syllabus) and of course, they have taken it out of context and gotten it wrong. And my definition of exploratory testing has also been plagiarized and corrupted (page 51). It’s mischaracterized as a cute little informal test technique when in fact it is an approach, universal to good testing, that applies to any technique, and the citation they give (to a chapter I wrote in an out of date book) directly contradicts what they say about it.]
Well, Carol stood her ground, at least physically. But as the debate developed, she made an odd-sounding claim. She said words to the effect that the ISTQB Syllabus is what it is, and is not subject to her criticism. This shocked me, especially since this admission was made in a rather off-handed way– somewhat like commenting that the moon is in the sky.
The reason for my shock was the sudden realization that I was putting a lot of energy into arguing with a person who was, essentially, behaving as a puppet. Oh dear. I was doing the equivalent of yelling at my television instead of engaging the guy ON the television.
If someone defends a principle that he has not originated and is not free to change, reject, or even criticize, then he is not defending it rationally. He cannot. No rational defense can be made under those circumstances. Rationality, in fact, loses its meaning. What he is doing is simply advertising his commitment. And that has no more weight in an argument than have the words of a baseball (or netball) fan who predicts his team will win the big game.
This triggered a niggling memory in me, and afterward it popped fully to mind: the Euthyphro Dilemma.
The dilemma occurs in the Platonic Dialogue of Euthyphro. Socrates is examining Euthyphro about the source of the notion of piety, or good behavior in humans. Euthyphro says that what is loved by the Gods (all of the Gods) is good. Then Socrates asks:
“Just consider this question: Is that which is holy loved by the gods because it is holy, or is it holy because it is loved by the gods?”
Now this isn’t just a question, but also an attack on the whole notion of appeal to authority. That’s why Carol’s offhanded comment triggered the memory.
Let me apply it to the case at hand: Is the ISTQB syllabus good because it’s a powerful, helpful set of ideas that define genuine testing professionalism, or is it good simply because the ISTQB organization says so?
This question is a dilemma for ISTQB supporters, because if they go with the first option, then they must:
- Avoid making any claim or behaving in any way that suggests they believe the syllabus just because they are unable or unwilling to study testing for themselves. (Saying that the ISTQB Syllabus was beyond her criticism violates this one.)
- Be prepared to explain and justify any aspect of the syllabus on when challenged by a colleague, just as anyone is normally expected to do for a personal opinion in a professional context. (I felt that Carol did not do this. At one point I cited my definition of coverage, and she agreed with it. Then I began to point out implications of my view that contradicted ISTQB dogma about measuring coverage, and I didn’t hear a coherent response after that.)
- Be prepared to change their views (thus breaking with the ISTQB) in the face of compelling evidence or reasoning. (I did not witness Carol do this, but I can understand that she wouldn’t necessarily consider my word as a tester to be evidence. What I can’t understand is why she seemed unaware of the work that my community has done and published, over the years, that does comprise compelling argument and evidence. This is not a new or novel debate. The issues have been clearly and repeatedly and publicly established.)
- Admit that they don’t need the ISTQB to learn testing, nor to be recognized as a good tester. (I don’t think this applies to Carol, since my understanding is that she doesn’t consider herself to be a tester, strictly speaking.)
And if they go with the second option, then they are choosing to be zombie non-combatants and can be safely ignored in the Great Conversation of testing.
Ideological commitment can be a bitch. That’s why, in the Context-Driven world, we keep that part pretty simple. There are seven principles that define our program. Of course there are many other common patterns and beliefs beyond those seven, but there is tremendous flexibility, because our whole focus is on DENYING a One True Way of testing.
We see testing professionalism as a matter of vigorous personal study and development. We reject any universal syllabus of testing. That’s why it hardly matters whether some particular definition or claim in the syllabus is also one I hold– because the nature of my commitment and the way I understand is completely different from that of an ISTQB board member.
This is why I say that supporting the ISTQB is, in and of itself, inconsistent with the goal of being a testing professional. A professional tester must own his craft.
Hi James!
Reading this post I got curious about a specific passage.
Could you please clarify how exactly your coverage definition implications contradict the coverage measuring practices recommended by ISTQB syllabus?
If you have answered this question elsewhere, I apologize and ask you to kindly point me to it.
Thanks in advance and best regards!
•••
PEDRO FARACO
Rio de Janeiro, Brazil
[James’ Reply: The short answer is that test coverage cannot coherently be quantified, except in specific, narrow, carefully qualified ways. These measures, if they are used, must be composed qualitatively into a meaningful and non-misleading story of testing. They should not be presented without context (an ethical standard) because they are not meaningful without context (an epistemic reality), nor can they be composed into one “super” number that represents coverage (since that takes it out of context). At least– not without violating measurement theory. Furthermore, we need not use any quantified measure of coverage, and in most respects and situations a professional tester should not (another matter of ethics).
My attitude about this is grounded in my study Naturalistic Inquiry, Qualitative Research, Grounded Theory, and the history and philosophy of Science. It’s not an arbitrary invention, but represents the paradigm in which I dwell.
And yet in the ISTQB syllabus we find “A variety of metrics (numbers) and measures (trends, graphs, etc) should be applied throughout the software development life cycle (e.g. planning, coverage, workload, etc). In each case a baseline must be defined, and then progress tracked with relation to this baseline.”
Hence, I reject that the use of the word “should”, which refers neither to a matter of ethics, practice, nor to descriptive observation. It’s reckless of the the ISTQB to blithely state that we should use metrics. I also disagree with the final sentence. Certainly we are not required to “define baselines” for our measurements, if we are using these unscientific and arbitrary measurements for their correct purpose: which is to learn about the project and about ourselves, rather than to control the project by outsourcing our stewardship to a formula.
By discussing coverage, I hoped to show Carol that what she thought was common ground between us is just an illusion. But I fear that particular pea was under too many mattresses to disturb her slumber.]
As a New Zealand resident of 11 years standing I can assure you that many New Zealanders love impressive sounding qualifications.
Got CISCO? You must be good. Got MS? You must be great.
The use of [fancy sounding] paper qualifications here as a shortcut to working out if people really know what they are talking about is common.
My experience of those with fancy sounding certifications is not so good, so I like to ask experiential questions when I meet such people. Theoretical knowledge is useful. I have tons. But as physics has proven time and again, practice beats theory every time.
Many in the certification business make a living from it. Pointing out flaws in such systems is not going to help them make that living.
[James’ Reply: What about a certification in Socratic studies? Socrates is constantly making fun of people who seek the wisdom of authorities.
“If I had not been poor, I might have heard the fifty-drachma course of the great Prodicus, which is a complete education in grammar and language- these are his own words- and then I should have been at once able to answer your question about the correctness of names. But, indeed, I have only heard the single-drachma course, and therefore, I do not know the truth about such matters; I will, however, gladly assist you and Cratylus in the investigation of them.” — Socrates (in Plato’s Cratylus)]
There’s that saying: “If you could argue with religious people, there wouldn’t be any religious people”. I think it easily applies to people who “believe” in the ISTQB syllabus. You can’t argue with them. You know it, and, sadly, they know it too – so they will just refuse to do it.
[James’ Reply: So, this hopeless situation leaves me only three reasons to argue, then.
1. To determine if they actually believe in it. (testing)
2. To practice and refine my explanation of my own position. (personal transpection)
3. To assist in the education of onlookers. (third-party transpection, case study, or exposition)]
“If someone defends a principle that he has not originated and is not free to change, reject, or even criticize, then he is not defending it rationally. He cannot. No rational defense can be made under those circumstances. Rationality, in fact, loses its meaning. What he is doing is simply advertising his commitment. And that has no more weight in an argument than have the words of a baseball (or netball) fan who predicts his team will win the big game.”
I am afraid while I agree with some conclusions you make, passages like the one above are just flawed in conclusion.
Your analogy – If a fan has a great deal of knowledge built up (as some do) about the skills and tallents of the players and their abilities as a team and the weaknesses of the other players, then I believe he can make a convincing arguement about the teams ability to win.
[James’ Reply: Yeah, whatever. You’ve kind of missed the point of my statement, haven’t you? Are you aware that, under normal conditions, the kind of people whom we would normally identify (or would identify themselves) as “fans”– as opposed to “analysts”– aren’t actually arguing at all, but merely expressing faith in their team? Do you get that? Or do you think that normally people who would scream “CRUSADERS SUCK!” are expressing a reasoned position on the issues?
Please attend to the point I’m making and the character of the evidence I’m putting forward, which is as common in your experience as in mine, I’m sure. Yes, it may be possible that some supporter of a team “crunches the numbers” and has an algorithm that somehow objectively shows the probability of one team winning against another. But that is not the sort of situation I’m talking about.
And for the record, no one has “crunched the numbers” to show the superiority of the ISTQB approach to professionalism.]
If you leave this as an arguement. You flavour it with infamatory words like “predict”, agreed its no crystal ball – but few arguements in software testing are either.
[James’ Reply: Do you mean inflammatory or defamatory? I suppose you mean both.
I’m flavoring it this way because I have contempt for them. You should also have contempt for them. Please raise your standards.]
Armed with well informed documentation about a companies manufacturing techniques I could argue that they are making valid contributions and are not adhoc or uncaring. I dont need to work for them to argue, not have influence to do so. If you choose to challence me perhaps we can both learn something we did not know. If you do so with a motive to changing the company then it is you motives that are incorrect, more fool you.
[James’ Reply: I don’t see your point. What does this have to do with anything I wrote?]
So
If someone defends a principle that he has not originated and is not free to change, reject, or even criticize, then he is not defending it rationally.
DISAGREE – anyone can rationally defend a process it if they have knowledge of it, they do not need to have invented it, own it, or have control over it.
[James’ Reply: You say you disagree. But are you arguing freely, or are you arguing under compulsion? Are you a chatbot or are you human? Are you capable of choosing your opinions, or have they been imposed upon you?
If I write a program that goes like this:
sub argue
{
print “DISAGREE – anyone can rationally defend a process it if they have knowledge of it, they do not need to have invented it, own it, or have control over it.\n”
}
Is that subroutine making an argument when I run it, or merely behaving according to its nature?]
I totally see your Point James. I compare it very much to my experience in matters of faith. For many there’s a time when Children may say I believe X or Y, they take it as a matter of faith, but if you pressed you certainly get the appeal of authority. My parents believe thus, or my pastor believes such, or my teacher/coach said thus, or some organization I belong to says thus. There’s a point at which we may blindly take something at face value, largely because of some trust in the individual or institution that the ideas come from.
However, for me, there came a point at which I matured, and by that I mean, there came a point when I no longer stood on other people’s arguments, but dug into ideas, and tried to fashion my own thoughts and motions. This is what I thought, or what I believed now, and though perhaps influenced by people or organizations in my past, as my beliefs grew, I didn’t see myself in the light of just those two, people or institutions.
So I agree with an earlier poster, that this very much sounds like a faith appeal to an authority, even though if pressed they might not provide a rational understanding of why things work as they do. It could be a sign of a whole in learning, or an area not studied in depth, and for certification schemes like the ISTQB, that to me is what makes it so ‘sinister’.
WOW you took offence like I was defending ISTQB – I am not.
[James’ Reply: I’m taking offense at your obtuseness. I think you’re not being as smart as you are capable of being. I don’t see how my argument is flawed. Instead, you appear to have fixated on a bizarre interpretation of my words.]
What I am defending is the need for those that argue against it not to use flawed logic – there is plenty of good reasons.
[James’ Reply: You don’t need to defend that, because no one is attacking that.]
If ONE fan can argue then the inference is invalid.
[James’ Reply: I was presenting a vivid example of a common human foible. The power of the example is in how accessible the situation is to the reader and on the improbability of the situation bearing more than a surface resemblance to rational analysis.]
If ONE person is knowlegable about ISTQB (but not in control of it) then thay can argue its benifits, and your inference is invalid.
[James’ Reply: This is a different issue. The first example was meant as a vivid and somewhat amusing example of typical irrational behavior. This was intended to get the reader into the territory where it’s easy to see my point. For some reason you were instead desperately searching that area for exceptions… But now you seem to be referring to my actual argument, which is that rational analysis cannot occur in the absence of freedom of thought.
I’m saying the very idea of rational debate has no meaning if the people engaged in it are not free to form their own opinions. That would not be rationality– that’s just marketing (or war).]
I am not proposing their arguements are valid or invalid, justified or not.
What it appeared to me from your post was that she was probably argueing the benifits, while you had a hidden agenda of changing not just her view but also influencing the ISTQB itself.
[James’ Reply: She wasn’t arguing benefits. She wasn’t really arguing at all. That’s my point. It was the pretense of arguing.
My agenda is hardly hidden. I’m not interested in changing the ISTQB. I’m interested in destroying it. It’s a fundamentally corrupt institution. It encourages people to conform, out of fear, generally, to a shockingly low standard of thinking and study.]
Sure in that endevour you would be wasting your time.
But consider if your thinking was flawed or lacking in some area of ISTQB and you did learn something more about it – then the arguement had value. If you pointed a weakness and altered her view – the arguement had value.
[James’ Reply: The premise of my article is the realization that no pointing out of weaknesses could possibly alter her position, since her position is not dependent on the strength or weakness of any view.]
If you considered your arguement the equal of “yelling at my television instead of engaging the guy ON the television” while there is a real person in front of you then perhaps you need to rethink. The Guy on the TV is just a news reader or Actor too. Engaging them would be no different.
[James’ Reply: Again, wow, it doesn’t matter, in my analogy, whether there is one level or nine between me and whomever actually does the thinking. The point is that it’s futile to argue with someone who is not in control of her thoughts.]
“The point is that it’s futile to argue with someone who is not in control of her thoughts.”
Yet you wouldn’t engage discussion if there wasn’t the slightest believe that she does have some control of her thoughts. Or did she engage discussion?
[James’ Reply: As I wrote in this blog entry, I didn’t realize until near the end that I was dealing with a Euthyphro situation. Now I think, all along, I was arguing with a puppet.]
Nice article.
The following point you make has come up in more than one ISTQB / exploratory testing discussion: “And my definition of exploratory testing has also been plagiarized and corrupted (page 51). It’s mischaracterized as a cute little informal test technique when in fact it is an approach, universal to good testing, that applies to any technique, and the citation they give (to a chapter I wrote in an out of date book) directly contradicts what they say about it.]”
The term ‘exploratory testing’ is often used by IT professionals I’ve met (usually project managers, developers, and sometimes testers) as a way to describe testing which is conducted in an informal (documentation / preparation light) way.
[James’ Reply: Yes, and that is probably appropriate, according to the two men who introduced the field to the term “exploratory testing” (Cem Kaner and me). Cem coined the term, first published it in 1986, and first used it online in February of 1995. I was the first to teach a class on it and to speak at conferences about it in 1996. Cem and I would agree that it is appropriate to describe informal testing as exploratory. But… that’s neither the best way nor any definitive way of describing exploratory testing. People who call themselves testing professionals should know that. People who actually are professional DO know that.
I specifically complained about this to people involved with ISTQB, years ago, and the description they use continues to be wrong.
Why is the ISTQB willfully promoting a misleading and decapitated idea of exploratory testing, instead of using the best possible examples and definitions as established by people who have devoted themselves to developing, refining, and teaching this approach? Never mind, I know why: they are incompetent and fearful people. Defend them if you wish. Does that elevate our craft? Does it help the ISTQB live up to their own code of ethics?]
This is usually when they’re running out of time and / or money but their business process requires some form of sign-off from testing, so they ask for testing to be done minus the red-tape (and often minus any actual testing) and call this ‘exploratory’.
[James’ Reply: Yes, isn’t that cute that they use such a big idea for such a small purpose? I wonder where they heard “exploratory testing” from? Did they read it on my blog or in Cem’s book? Did they hear from one of my talks?]
Getting back to the quote from your article, my impression is the ISTQB definition (by use of ‘informal’) reflects a common perception of the term’s meaning… whether that is their intent or not I have no idea.
[James’ Reply: I thought they fancied themselves a professional organization. Does it make sense for them to feature, in their very syllabus, poorly researched rumors about industry practices? What does that say about all the other material in their syllabus? Can we trust that any of it is based on good scholarship?]
In either case it bares little resemblance to context driven practices, short of the name.
[James’ Reply: Exploratory testing is not a context-driven practice. I’m not sure there is such a thing as a context-driven practice, as such. ET is an approach to software testing used by all schools of testing thought. Some communities do it better than others. The Context-Driven community has a special affection for it, I suppose.]
“Just consider this question: Is that which is holy loved by the gods because it is holy, or is it holy because it is loved by the gods?”
Now this isn’t just a question, but also an attack on the whole notion of appeal to authority. ……………………….{ I didn’t see how , regardless }
[James’ Reply: Don’t worry. Maybe philosophy is not your thing.]
Let me apply it to the case at hand: Is the ISTQB syllabus good because it’s a powerful, helpful set of ideas that define genuine testing professionalism, or is it good simply because the ISTQB organization says so?”
I like it… you could almost continue your argument and tell interviewers it’s an utter waste of time interviewing candidates for the post of tester, because after all are they a good tester because their experience has taught them how not to make so many bad decisions, and to work efficently clearly and coherently, and that any piece of paper be it a university degree, an ISTQB certificate or a CV that doesn’t give you any of this information that makes them a good tester, which you would require to give them the job? So hire any old dude or dudette and let them learn on your time and dollar right?
[James’ Reply: Um, no. Hey, if I liked you more I would refuse to publish this until you sobered up and had a chance to rethink.]
You seem to be arguing that testing is so individual, the how to do it, cannot be quantified, measured, taught or tested, not by the ISTQB in any road.
[James’ Reply: No, that’s not what I’m arguing, although it’s true that testing cannot be quantified in any meaningful way.]
In short while you obviously have an axe to grind with ISTQB whoever they are, I would caution about throwing out the baby with the bath water……personally, while I partook in the 3 course, and in a foreign language, and didn’t really find it that useful, I have found more and more employers holding it up as a standard something or other.
[James’ Reply: I do have an axe to grind with bullies and frauds. Don’t you? Are you one of those people who can watch crimes happening in the street and just keep walking?
Yes, many employers hold it up as a standard. They are idiots. The good news is that you don’t have to be. It’s a choice you are making, but you can make a different choice.]
To my way of thinking, half an anything is better than a complete nothing… while doubtless ISTQB can develop… I would like to hear, how we can if at all possible start talking positively about the world of test as it applies to the workers… as it applies to me, rather than every site knocking ISTQB and every employer demanding it..
[James’ Reply: Most of my body of work is exactly that. Perhaps if you read some of the things on my website, or in my books…? Try reading more of my blog…
However, look, I don’t know how old you are. I assume, since you think that the alternative to ISTQB is “nothing” then you must be pretty young. Son, the world is a rough place. Sometimes people do bad things. When they do, men of good will should step forward and oppose them. We need to oppose the ISTQB, so that the bad men don’t destroy the reputation of our craft.]
How if your experience is so vastly differently to mine, could either of us ever employ one another to test?
[James’ Reply: No. I only work with people who are critical and skeptical. And coherent.]
James,
Thank you for this post – I find the link back to Euthyphro interesting and hadn’t thought of it when applied to testing dogma.
I continue to be astounded at the lack of understanding of what “Exploratory Testing” is amongst people describing themselves as ‘testers’. When ET is criticised for being ad-hoc or difficult to trace back I have started responding with “it can be – but it can also be highly structured – it depends on the needs of the business and the individual project at that point in time”. To me that is the beauty of the approach – it adapts to the continuously changing context.
To apply Euthyprhro good testing is good because it actually meets the needs of the business and project at the time NOT because it says it does.
Stephen
[James’ Reply: The dilemma was a dilemma for Euthyphro because he couldn’t answer the argument “well, why do we need gods, then?” But in our case, we can answer that: we don’t need the “gods” of ISTQB. We can come to our own beliefs and defend and modify them as we go.]
James, your aptly formed ISTQB dilemma, sounds a lot like a complex question in the form of a dilemma. This is just like asking, “Have you stopped beating your wife?” It doesn’t matter which answer you give, you are incriminating yourself.
[James’ Reply: Obviously, in that case, if you are given a question that contains an invalid premise, you reject the premise (e.g. “I’ve never had a wife. Perhaps you have confused me with someone else?”). The problem you have is not a matter of incriminating yourself, but one of sounding evasive (e.g. “Seriously, man, I’m like eight years old. No, I don’t have an ID, for crying out loud.”).
Rejecting premises sounds evasive because A) most people have no patience for the philosophical process, and associate it with “bullshitting”, and B) bullshitters do a lot of rejecting of premises as part of their art.
The Euthyphro dilemma is a real dilemma for people who wish to rely on authority in an argument. If an authority’s command must always be accepted, regardless of what it is, then we must also accept that our own judgments about the quality of such commands are irrelevant (that’s the “sock puppet” problem– you look like a human, but you’re just a mouthpiece for someone else). But if the authority itself is behaving non-arbitrarily (that is, according to some rational or otherwise patterned system for making decisions), then those determinations are subject to review and criticism. Our own judgments do matter and the question “why do we need the authority?” becomes important.
Euthyphro didn’t want to admit that the gods might be wrong, and he didn’t want to admit that the entire basis of morality is the caprice of deities he can’t even communicate with. He was trapped.
I am not trapped this way, because I don’t accept that there are authorities in testing. There are people I respect, and they may say things that I take very seriously. But I don’t accept anything in my engineering practice unless I– personally– can explain and defend it.
That’s how I teach, as well. I don’t tell anyone “this is how to answer the question when you see it on the test so that the Oligarchs will pat your head.” Instead I invite students to come up with better ideas than mine (and then I add those to my class!)]
The case of Ms. Cornelius is somewhat typical because she sounds like she is just doing her job for the sake of doing it. Apathy perhaps? I must admit that I consider myself as a youngling in the Context Driven world. I am and will probably always be questioning why I believe certain testing concepts. When I started in the testing field I took every single training that was thrown my way and that was really the foundational basis of my testing career. The point that I’m really getting to, is that most new testers get fed wrong information because their companies force fed them the wrong testing ideologies, and certifications included. The ISTQB Marketing arm is the force that’s driving ISTQB’s level of success and unless they run out of customers, they won’t be out of business.
I thank you, Michael, Pradeep and the rest of the context driven veterans for doing a great job in spreading the good word about what testing should be. Isn’t it about time the Context Driven School start conversations with the decision makers of corporations so tester unschooling can be accelerated?
[James’ Reply: I’m working with two large international corporations right now: Barclays Capital and Progressive Insurance. They have committed to creating a professional testing culture based on Rapid Testing and Context-Driven Testing principles. I’m close to getting permission from Barclays to talk publicly about what we’re doing there.]
Dear James, readers
I find the discussion utmost interesting and it even made me question my own thoughts!
When ISTQB is talking about measures, they sound a lot like Peter Farrell-Vinay. I am not sure if that is a correct assimilation, but it sounds like what I recall reading from “Manage Software Testing”. Considering, ISTQB would be a teaching organization, I would like it to say why those measures should be done. (In general, people don’t need to be told what to do, but why something should be done and they will figure out a way that suits them best.) I have done some metrics, and definitely will keep doing them if they help me to communicate with upper management, but recently I have started changing to more qualitative information, such as discussion with the testers about possible risks and their best bugs. I must note, I am in the domain only for a few years, so I don’t want to sound like I know a lot about testing.
There is a lot of criticism in the comments regarding wordings, analogues and arguments. Surely those are important things to question! (Even I noticed a few.) But, I think the main point is what James is saying, not how he is saying it! Of course he uses humor and other controversial techniques – and that is one of the reasons we are reading this blog. Valid observations presented and decorated in an amusing format.
For example I agree with this comment “Armed with well informed documentation about a companies manufacturing techniques I could argue that they are making valid contributions and are not adhoc or uncaring. I dont need to work for them to argue, not have influence to do so.” However, this is not what was said, because James said “to change, reject, or even criticize”. I understand companies could hold a lot of information about a whole lot of things, but this doesn’t mean the information is categorically correct. I see this as something which differentiates inventors from other people. This is why James started teaching exploratory testing (which requires the tester to innovate while working instead of executing pre-made test cases).
[James’ Reply: My point was simple, really. If I’m going to have an argument, I should address my opponent at the appropriate level. It makes no sense to debate the merits of the U.S. Constitution or local Penal Code with a policeman on the beat. The guy has no choice but to follow his instructions.
Certainly you can defend something that you don’t control, but NOT while are CONTROLLED BY THAT THING ITSELF]
What I don’t seem to understand is, many people keep calling “free testing” as exploratory testing. (I use quotes because I have no idea if some organization has defined the term already.) Exploratory testing term is used when testers can do what ever they want, managers/leaders/… don’t want to a) assign test cases b) create test cases c) report “test case coverage”, etc. Why they feel the need to call it exploratory testing? Does it ring nicely on the back of the head? My claim is when you do what you want, when you want, how you want, … not following any process, you are doing something I call “free testing”. I am sure someone with better English can come up with a cooler name. I admit, aforementioned can be included in exploratory testing, but that is like telling a keyboard is a computer. (Yes yes, bad analogy, sue me. 🙂
[James’ Reply: By “free” do you mean self-managed? Exploratory testing is indeed a self-managed process. But in naming it we focused more on the dominant cognitive process (which is important because that’s where the value of this approach derives from), and not the presence or lack of outside compulsion.]
I find it funny James wrote “But I don’t accept anything in my engineering practice unless I– personally– can explain and defend it” because I think this is exactly the reason why there are so many ISTQB believers – they can’t explain nor defend it properly. Call it what you want, it’s some sort of herding and it’s really common in life. Another bad analogue: go and ask a (young) person why s/he thinks the latest Britney Spears song is so great. Clap your hands to make a beat for the reasoning.
As a curiosity, I have an ISTQB “certificate” since a few years ago. All our company’s testers were prepared to get the paper because apparently many customers think it’s a must and adds value for our testing business. Due to that, I know ISTQB Foundation Level certificate is about understanding the vocabulary (actually just having a common one) and as James has clearly shown, not doing even that great job in it. What I personally think is the worst thing in the certificate is not there are obvious problems, but it tells you “must” and “should” everywhere instead of challenging to get new ideas and ways. This is the worst thing to do for a creative mind.
[James’ Reply: Thanks for the insightful comment, man.]
I think it’s unfortunate that people and the businesses they wish to be employed by are slowly becoming stuck in a wheel of expectation to conformance. Businesses expect to see test qualifications, regardless of whether the interviewer has any clue as to the content of those courses, and the interviewee knows that having those pieces of paper will help them get the job, so they conform. I don’t necessarily want to have to take courses as I advance up the ladder to prove myself to future employers, when my ten year CV and my ability to be questioned on my experience should suffice. But, already I am finding it is expected, with little convincing explanation of why, by recruiters who don’t even understand themselves. Recruiters are even searching CV’s by keyword alone more and more now, so if those letters are absent, you may not even get noticed.
[James’ Reply: I think you are looking at this problem the wrong way. See my book Secrets of a Buccaneer-Scholar. It’s about how I got around this whole issue. I established a reputation. My reputation causes people to recruit me. I don’t show up to strangers and ask them for a job. Neither do you need to.]
It is this kind of pressure that makes people take the course in my experience, rarely because they think it is actually essential to their career, or even want to. I don’t see developers having to take tests in advanced programming to get a job as a senior programmer. Why is this? I have also interviewed and rejected many candidates with and without test qualifications, having it has proven no assurance at all so far to me, I don’t use it for CV filtering therefore.
I’m going to take an advanced course soon, again because I feel occupational pressure to, but also, I will feel in a better position to question those who have if I have done the same process and already know it’s strengths and weaknesses. I am sure it has both.
[James’ Reply: You don’t need to eat with pigs to know why food hygiene matters. You don’t need to steal to understand why stealing is wrong. You don’t need to compromise on this. Stop feeling weak, because you aren’t.]
I am an ISTQB certified tester. The greatest thing I learned from ISTQB certification is “I can crack another thousand of such certifications”. In fact everyone in the planet earth can do it.
Probably the biggest challenge in clearing one such exam would be the rigorous mugging up (you read it right, mugging up, you can’t afford to disagree or understand for that matter if you really want to pass the exam) of the syllabus.
If they are candid, all who passed the certification would agree that there are questions where they have, if applied their experience or intuition answered it wrong. They know the answer per ISTQB for such question is wrong but they couldn’t afford to choose the answer which they think is correct for they would fail the exam otherwise.
One such disagreement of mine is they have put “Exploratory Testing” on purely experienced based testing. James, correct me if I have got ET wrong (and I am always ready to be corrected if I have misunderstood something), a novice tester who has got great cognizance and sapience and can explore things better, can think of different ways the product may fail to perform per requirement can always do a great job in ET than a 5 years experienced tester who has only learned to execute a set of test cases. That is probably one of the beauties of ET. There is of course, always an advantage of having some experience but that alone doesn’t suffice ET to be put under experienced based testing.
[James’ Reply: I’ll reply to this in the form of a blog post.]
I loved the comments section and read it with equal alacrity for it has some fine arguments. Those who are adamant to change their preconceived wrong notions about something can be benefitted from this. How beautiful an unbiased argument can be!