An open letter to James Whittaker:
You wrote: “I had an amicable hallway conversation with James Bach. His blogger angst at my use of the title ‘Exploratory Testing’ didn’t spill over to a face-to-face discussion. Frankly, I am not surprised. I’ve never claimed the term as my own, I simply took it and made it work in the hands of real testers on real software under real ship pressure. Consultants can coin all the terms they want, but when us practitioners add meat to their pie, why cry foul? Is it not a better reaction to feel happy that there are people actually doing something with the idea?”
None of that is true.
I would not describe our conversation as amicable. Perhaps you thought it was amicable because we didn’t talk about anything important, and during that moment, I didn’t raise my voice. Or punch you.
My criticism of you is not “blogger angst”, it’s my opinion based on studying for 20 years something you’ve hardly studied at all. Every substantive conversation we’ve had has consisted of you denying whatever I happen to say, without offering evidence and in most cases without offering an argument. You have a zeal for dismissing my work that is truly extraordinary– you once even denied, again without evidence, that I knew how to run a file compare tool. Wow.
Now you say you made ET work? Well, first, you don’t know what ET is. Second, you’re an academic. You stayed in school and studied formal methods (that no one uses) while I was cutting my teeth in Silicon Valley. I have taught and demonstrated ET all over the world. I’m not alone, but work with a community of like-minded testers and thinkers, comparing notes with them, and deepening our understanding of exploratory learning applied to testing. You have not been a part of that.
I don’t think you’re adding meat, I think you’re serving thin gruel.
ET does work. My community repeatedly shows that it does. We will patiently continue to teach and develop it.
Subtlety hasn’t worked with you. So I’m saying this publicly: You’re a good speaker, but as a practitioner, if your prose is any indication, you don’t know much about what you are doing. If you applied yourself, you could become a good tester. But I’ve seen no evidence of that, yet.
Sounds like the two of you should have an open testing duel and settle this on the field. Pick and app and see who can find the most bugs in an hour. Or better yet, give each of you a team of 5 kids from a local college. You have to use that team in the challenge. This would give you a chance to prove, leadership, teaching and execution.
We could even get this on some kind of pay per view event, or make it a video presentation a future Star Conference.
[James’ Reply: That’s why we’ve run testing competitions at the CAST conference.]
Moving on from that idea to another. The main reason this is such a big deal is because the testing community at large is not smart enough to see through a book? I guess I don’t see a similar type of dual occurring in the software construction field. It seems that the main angst beyond the personal battle, is that because James W claimed something the fear is people will latch on to that and take it as gospel because it was printed. This is similar to the fear with ISTQB certifications.
[James’ Reply: I’m not worried about that. The reason I wrote the post is three-fold: First, Whittaker wrote as if I endorse (or do not oppose) him, when in fact I don’t endorse him and believe he is not doing a competent job. Second, as one who has done seminal work in this area, my opinion is of interest to some people. Third, since my hints to this effect have been ignored, I wanted to publicly call on him to do better.]
So the real problem lies an immature field of practitioners? Where as in a future world where everyone is smarter/educated/something, it wouldn’t really matter much because no one would need to be told not to buy the book.
[James’ Reply: Reputation matters in our field. People should suffer or prosper by the quality of their work.]
I’m surprised you didn’t mention this from further down in his post:
“(Has anyone coined the term ‘automated exploratory testing’ yet?)”
Though, I’m not sure there is a response to this 🙂
[James’ Reply: Exploratory testing is a sapient process, so the prospects of automating it are limited– unless you have a very simplistic idea of what exploratory testers do, which is exactly one of my concerns about Whittaker and Rollison. However, explorers may certainly use tools to aid them. That’s the basis of my Rapid Test Automation concept.]
It wasn’t right for him to call you out by name and misrepresent you on his own blog as someone who isn’t a practitioner, just a blogger and/or consultant.
But you take a few swipes of your own and I hate to see people writing from anger or frustration; life’s too short and I’ve seen a few such blowups hurt the emacs community, the Ruby community, etc.
At the risk of sounding like some clueless hippie — is there some mutual friend you share that could help mediate this dispute and help you find a common ground, where you “agree to disagree” about things like the correct meaning of “Exploratory Testing” and what approach to testing is best? I.e. not a matter of right and wrong, so much as different schools of thought and practice, and no more personal attacks.
[James’ Reply: Sometimes personal attacks are justified. It’s unfortunately the case that there are charlatans in our industry. Should we just ignore them?]
When I read this post I was struck by how upset you seemed to be with Whittaker. I then read your post entitled \RECLAIM YOUR PERSONAL METHOD\ from June 4, 2009 and I could not help but think that this post seems very hypocritical. You may not approve of the way Whittaker is speaking and writing about ET but perhaps he is just making it his own as you recommend in your June 4 post.
[James’ Reply: I’m not criticizing him for making it his own. I’m criticizing him for doing bad work in an area where he damn well knows better.]
Sorry for last post, please ingore it. thanks!
First cite your words in (the last page) A man approached me at a conference and said that he ran an exploratory test team for a large telecom company. He said his team moves from project to project, and the reason he’s allowed to continue his work is because their metrics show that his team find four times as many problems in a given period of time than do scripted testers.
And my question is that could you give me the detailed metrics which showing the productivity of ET testers or how can the guy calculate the “four times” ? the similar question is that what the metrics to be while I suggest my test manager to consider me to build the ET team ? and as his examples when the ET team get involved with these projects and what time period is ?
[James’ Reply: I don’t trust the metrics from that fellow. I didn’t collect them myself. I report what he said, but I can’t endorse it. This is a frustrating thing about hearing second hand reports.
The metrics I use come from Session-based Test Management. You can read about that on my website.]
Wait, wait, wait… Did he actually say that “he made it work in the hands of real testers on real software” and “that there are people actually doing something with the idea?”…
I don’t know Mr. Whittaker, so I checked around a bit. According to LinkedIn
– 3 years w/ IBM as consultant
– 10 years teaching
– 3 years at Microsoft as ‘Architect’
– less than 1 year at Google as Test director
So, if my math serves me, that’s maybe 6-7 years actually in the test arena?
He says ” I’ve never claimed the term as my own, I simply took it ” That is one of the things that is wrong Mr. Whittaker. It is gentlemanly and honorable to acknowledge the credit where it is due. What it seems he did is a not-so-subtle form of plagiarism.
Thanks James, we look to you and the rest of “the Guys” to keep fostering our learning.
@Jerry Gao —
>>> And my question is that could you give me the detailed metrics which showing the productivity of ET testers or how can the guy calculate the “four times” ? the similar question is that what the metrics to be while I suggest my test manager to consider me to build the ET team ? and as his examples when the ET team get involved with these projects and what time period is ?
Why is that everything that we do in testing has go through the path of metrics? If I want I can make “four” times to “eight times” – it is all about tweaking numbers right? I can club 5-6 bugs in one or make one bug into several bugs. Bugs are riefied entities – they manifest relationship between user and the application.
So do not believe (spread this meme… please) if someone says they can provide proof for any testing methodology/tool in terms of metrics – Do not believe them. It is a trap
Shrini
@James, Thank you very much for your reply.
It’s a big surprise to me for you don’t even trust the metrics.
I have read your another article “Session-based Test Management”, and think this metix is just for ET, I mean the data collected just for ET, the time collection etc. there is no evidence to verify we need to do ET in individual time period for manger’s viewpoint. could you help me to jump out it?
Thanks again.
[James’ Reply: I don’t understand your question. Sorry.]
@SHrini
Thank you very much for your reply.
I agree with your comments. but as the nomal tester, if he want to build the ET team, and do ET testing during working time, positively need the lead’s agreement.
And of course he want to know the reason, especially the metrix to show this ET approach is better than scripted testing, I think you should understand this true situation from the prospective of managment.
Also the question is also unsolved: how can show or persuade the manager allow me to do ET during working time rather than do scripted testing even build the ET team? it’s a big confusion for me now.
In fact, I am researching this ET approach and practice in my team.
Finally. I am absolutely trust the ET and will research it allways.
Thanks.
[James’ Reply: Just SHOW them how you find LOTS of BUGS with it.]
Some people seem to be offended by James’s writing. I sort of vaguely understand where they are coming from.
[James’ Reply: Sure, I understand, too. Or so I think. That’s the rub, eh? All of us think we understand. Maybe we do, maybe we don’t. In any case, we keep trying to get it right.]
Look, you can draw your own conclusions about Whittaker by looking at his Bio and Linkedin Page, and attempting to verify his claims. Just Try. It’s only a google search away:
bio: http://www.dotnetrocks.com/default.aspx?showNum=408
LinkedIn: http://www.linkedin.com/pub/james-whittaker/13/878/229
Now, if you do your own research, and come to the conclusion that a ‘thought leader’ is stretching the truth in order to trick people into listening to him – and you self-identify as a professional critical thinker – what exactly are you supposed to do? Stay quiet? Certainly not.
James took a risk posting this. I might use different rhetoric and have a different style, but I applaud him.
[James’ Reply: I don’t like having to write posts like this. Sometimes I feel there is no better option.]
@James, the first question is similar to another one.
As we know, there are lots of effects for ET, on other words, the bugs finded by ET also effected by lots of factors, for example, product itself, available time, available resources, the involved time during project.
so if I have the chance to do ET in my working time for other project which is unfamiliar with me, and the bugs finded is limited or very fewer, than we need to analyze the reason. but you know at this situation it’s very ashamful to me during ET. And also
be a big obstacle for promoting the ET approach in China.
So my question is that should we need to do the preparation avoiding this bad situation? and also give us some suggestions for our chinese testers.
Thank you very much.
[James’ Reply: Success with ET is based on the fundamental skills of testing. I can’t tell you how to be successful with ET without going through the process of teaching you testing.]
[James’ Reply: Normally I don’t accept anonymous comments except to mock them. I’ll take this one because you raise a good point that I want to respond to.]
Slightly confused. My issue with this post is that it seems rather incomplete, and incompleteness makes it seem dishonest. Isn’t this the same James Whittaker that Lessons Learned in Software Testing recommends on occasion, the same James Whittaker that received a five star review from you on one of his books?
[James’ Reply: Yes, I gave his book a good review. As I said in my review “As a test designer, myself (and a competitor of Whittaker’s) I can certainly find things to nitpick about this book. But I won’t do that here, because the big picture is far more important.” The concern I was referring to was Whittaker’s simplistic view of testing. This was not so terrible, because in his book, he did not claim to be describing exploratory testing in general. He was offering some test design heuristics. Fair enough– although it should be noted that his book is based on ideas that were original to Alan Jorgensen.
I wrote that review seven years ago. Since that time, things have happened in the community. Whittaker torpedoed his own reputation by certain acts that I’d rather not rehash publicly due to the fact that I’m not at liberty to publish the incriminating documents– but he knows what I’m talking about. Meanwhile, we’ve made a great deal of progress in developing ET as a discipline. Whittaker has not been a part of that work. In fact, he sneers at it. Well, that’s his right, but I’m not going to let him imply that I endorse his sneering.
The post is the most minimal thing I could write. I don’t want to talk about Whittaker at all. I just wanted to go on record to counter a specific thing he said, and to let folks know that his book does a poor job of describing exploratory testing. My opinion is relevant to some people. Take it or leave it as you wish.]
All this seems to conflict with: “If your prose is any indication, you don’t know much about what you are doing.”
Given your previous view, the post seems to at least merit some sort of “His previous work was good. now not so much” or “I reassessed his previous work, I was wrong, it’s not that great. Here’s why…”, no?
[James’ Reply: His previous work did not use the term “exploratory testing.” It was narrow and shallow and that was okay. Now he’s trying to capitalize on a reputable idea made reputable by others.]
I read the book, but apparently somewhat differently than Adam Goucher. The book did not make my short list of must-haves, but I did not feel I wasted money or effort into the book. I enjoyed the tours as a description of James Whittaker’s classification of things he thinks you could do when testing, as a way of reminding of the variety of viewpoints. In his discriptions, he gave names to some of the things that have been hard to explain to new people in testing where I work, typically with a non-exploratory testing background. While the names say very little, reading the stories helped me remember most of the stuff after just one round of reading.
In addition to the tours, I liked the fact that he discussed exploratory testing as a technique. Even though I strongly feel its an approach, I work with a lot of colleagues I respect that would seem to never raise it from technique status. While the hybrid approach as Whittaker describes is less than the exploratory testing approach in value, its still of more value than just the scripts / scenarios.
I also found the annotated blog posts useful, especially perhaps since I had read the original blog posts when they were posted and changes of mind were useful.
I agree with the difficulty of publishing something better with the same title. However, the internet is also full of descriptions of ET as a technique in a more shallow way than Whittaker takes in his book. And then there’s the good stuff, like the latest post in this blog. If Whittaker’s book gets someone interested in reading more with the title (knowing the title still seems to happen all too rarely around me), there’s a good change they will expand to some of the good stuff by the ET community.
I’ve taken time to read thicker books with less content than this one. The best content to pages ratio in my opinion still goes to Kaner, Bach, Pettichord: Lessons Learned in Software Testing.