It would be an unsound fancy and self-contradictory to expect that things which have never yet been done can be done except by means which have never yet been tried.
This reminds me of that “definition of insanity” which is so often attributed to Einstein. But this comes from Francis Bacon, circa 1620. In his seminal work “The New Organon”.
Bacon has some other interesting quotes for testers…
The logic now in use serves rather to fix and give stability to the errors which have their foundation in commonly received notions than to help the search after truth. So it does more harm than good.
This is how I feel about the ISTQB syllabus.
There is no soundness in our notions, whether logical or physical. Substance, Quality, Action, Passion, Essence itself, are not sound notions; much less are Heavy, Light, Dense, Rare, Moist, Dry, Generation, Corruption, Attraction, Repulsion, Element, Matter, Form, and the like; but all are fantastical and ill defined.
This is how I feel about a lot of testing terminology.
It is idle to expect any great advancement in science from the superinducing and engrafting of new things upon old. We must begin anew from the very foundations, unless we would revolve forever in a circle with mean and contemptible progress.
I feel that way, too, about patterns such as the v-model, and most of what passed for test techniques in the 80’s.
Francis Bacon was proposing a great break with the stifling Aristotelianism of his day (the certification craze of the middle ages), and sought a new foundation for science. Bacon thereafter became a godfather of the enlightenment, helping to create the modern world.
We’re way beyond Bacon, now. Still, I’m attracted to his sentiment that what passed for good scientific work in his time was actually nothing but uncritical folklore. In our time, in our little field, we need a similar re-invention of the craft, a New Organon of testing.
James,
Do you think that this applies to all of software development or only software testing? Do you see software testing as a separate discipline?
My opinion, although I am not sure if you want it, is this. As a community we are struggling to break the foundations of our recent history. We have some strange co-dependency on many things and a penchant for the two Bs: buzz words and bullshit.
So, no more testing techniques. No more methodologies. No more frameworks. And, actually, not even a New Organon for testing or software development. Unless, your New Organon is a complete over haul of the way we teach, coach, learn, and communicate. Because these are, in my opinion, the biggest challenges we face. They are certainly the biggest I face on a day to day basis.
Anyone can develop software can’t they? I do it. But not everyone can teach it, or coach it, or prepare an organisation to do it. These are the challenges aren’t they? So, maybe we should stop sub-optimising on some things and start attacking the real bottlenecks. Once we smashed up the old foundation, the old teaching and the desire to short circuit everything in this ADD industry we survive in, maybe then we can get back to the business of optimising our techniques.
Called me old fashioned, but don’t we have to put first things first?
Tired. Going home now. Jamie.
[James’ Reply: If the old Organon worked, I would use it. But it doesn’t work. It never worked. Just like Aristotelian physics, it never worked. So let’s put first things first and stop doing things that don’t solve problems.
I know you a little bit, Jamie. What I like about you is that you are not confined by traditional ways of thinking. You are pragmatist who solves problems as you see them. That’s what I do– but I also practice talking about and teaching how I do things.
Francis Bacon was bucking a powerful hold on the minds of men. It was a system designed to control society by “anticipating” the truth, as he put it. I am bucking a much lesser thing, in this industry, but still it is ominous and oppressive. My goal is to resolve the alienation testers feel about their own thought processes.]
James,
Maybe there is no alienation at all but we are simply locked into reactive thinking based on what we have learned. For example, a colleague of mine, who I asked to test a feature, started by diligently writing a test document in Word.
Strange, I thought, since a document is not a test. I’d have probably just started playing with the application. I wonder, do testers and developers feel alienated from their own thoughts? Or do they simply not have thoughts of simplicity and putting first things first?
[James’ Reply: There are several things going on, of course. The one I’m most interested in at the moment is when people feel that their work processes don’t belong to them, but rather to some expert somewhere. This is the alienation I’m talking about: “I can’t own my choices, or explain them. I am doing things because of convention and not because I really understand the problem or the available solutions.”]Â
Or, more likely, they do see nice simple solutions but their fear gets the better of them? The fear may come from not wanting to look silly, or swim against the tide. Or even more likely it’s a combination of the two. And herein lies our problem as teachers and practitioners: how do we first help with the generation of novel ideas, how to we teach lateral thinking; and how do we nurture our colleagues to implement them, to find the courage they need to, seemingly, swim against the tide?
I know what I do and it involves a lot of nurturing, teaching, and bad language in sentences that sound a bit like: “I don’t give a flying **** how it was done in the past or about your bull**** procedures�. Although, I say all this with a smile on my face which seems to help. I am not in the job of frightening people! Compassion is actually at the centre of most of my work. However, this anti-authoritarian stance can really help with building confidence.
[James’ Reply: Yes. I’ve seen you practice this as well as preach it.]Â
What’s interesting though is how we get to this state. How did we get into a position where testers and developers buck responsibility? Forcing managers to create more BS processes, alienating the team more and continuing the cycle ad infinitum. This type of relationship is true co-dependency where managers define themselves via their lazy teams and teams define themselves via their incompetent managers and their awful, wasteful, processes.
Breaking this co-dependency, and similar relationships, such as your testers’ co-dependency on fear and novel thinking, is the real challenge I think. And it is bloody harder than teaching any one technique, methodology, or tool. That is our challenge my friend.
This co-dependency thing can be proved with the port test. Next time you are on a job ask to get a port opened on a server. Then sit back and watch how many forms need filling in, how many processes people hide behind, responsibility-absolving-processes I call them, and then measure how long it takes. The port test will give you an idea about how difficult your gig will be. How about that for encouraging alienation? Is there any wonder your people feel alienated? This is a feeling they are used to, and that is encouraged by our culture.
Think about some rough kids you may have known. Why was anger such a successful defence mechanism for them? Because it worked right? Helped them grow up, survive, et cetera. Now, years later these angry kids become angry adults. These adults treat every new frightening situation with anger. It is not the situation but rather the fear that kicks off this Pavlovian response. Thus, when your testers see something new, if they were brought up in a culture that promotes alienation, then sure enough alienation will be their first response. The human brain does not differentiate much between outside and inside stimuli.
So, when you say “My goal is to resolve the alienation testers feel about their own thought processes “ then your work is definitely cut out for you. Because it is not like teaching an equation. It is actually un-raveling learnt behaviour, which is usually the job of very skilled psychologists. The positive thing, of course, is that if we attack this as a community, from many angles, this is a problem we will solve. In our lifetimes, maybe in the next ten years. We don’t need to change everyone, we just need to create a critical mass.
Enjoy Edinburgh, let me know if you need a night out. I used to live there, my mate Adrian will look after you if you need it. And, stop labeling me as pragmatic please. You are ruining my reputation as a refined, British, academic! I am methodologist of the first order! I went to university and everything.
Jamie.
[James’ Reply: Yes, my work is cut out for me, but I have figured out how to do it (at least for some people). You saw some of it in the challenges I did around the table, at our first dinner. I get people doing something, and then I notice when they are clever and try to put a voice to that cleverness. In this way, I can reintroduce people to their own intelligence and help them begin to claim it and take a stand.]
For me the New Organon was the four volume Software Quality Management Series by Gerald M. Weinberg. It does not give testing techniques but lays out the background against which modern software testing can be effective in the industry. For modern test technique we need a Newton’s Principia – as it happens, an excellent start to something like that is “Lessons Learend in Software Testing”.
Cheers.
Thanks James.
What Bacon wrote nealy 400 years ago aptly applies today — and it applies to software development and testing. Bacon argued that placing our preconceived beliefs over what we observe causes great harm. He went so far as to describe these harmful preconceived notions as “idols”. Bacon put these idols into four categories:
* Idols of the Tribe: Errors common to mankind.
* Idols of the Cave: Errors specific to each individual’s education and experience.
* Idols of the Market Place: Errors formed through association with others — often due to misunderstanding others.
* Idols of the Theater: Errors formed from dogma (institutionalized doctrine) and flawed demonstrations.
All of these exist in software testing. As testers, we should be questioning these “idols”, not worshiping them. … and sometimes questioning them may confirm them. 🙂
Bacon did not ask anyone to abandon their beliefs without cause. Instead he asks that we not make them idols capable of leading us to ignore what would be obvious if we weren’t looking through the distorted mirror of our idols.
A modern day simplification of Bacon’s arguments may be the Agile Manifesto. We should not let our idols of process, documentation, contracts, and plans prevent us from accomplishing the desired goal. Process, documentation, contracts, and plans are only good in as much as they help. They should not prevent us from seeking improvement.
In some ways I believe that the promotion of testing folklore is the result of an industry-wide desire to show that we are mature — as mature as the engineering of physical products. That eagerness to demonstrate maturity leads to the implementation of bad processes and cerfifications. Ironically, enforced process works best for the immature and gives the impression that anyone that can follow the process can test software.
We need to seek continual improvement. It is sad that process and certification often become idols that overshadow the real goals.
[James Reply: I doff my cap, fellow traveller!]Â
You wrote:
“Bacon has some other interesting quotes for testers…
The logic now in use serves rather to fix and give stability to the errors which have their foundation in commonly received notions than to help the search after truth. So it does more harm than good.
This is how I feel about the ISTQB syllabus.”
As someone who is working at and with the ISTQB syllabus I wonder what annoys you with it? What are errors in it?
Maud
[James’ Reply: The ISTQB syllabus represents the views of a particular group of people as if they represented all testers. It is the product of a particular insular community, but not identified as such. Even if the syllabus coincided exactly with my own ideas about testing, which it does not do, I could not support it because there is no universal community of testers.
I think the syllabus oversimplifies testing to an outrageous degree. I say outrageous because the leaders of the ISTQB organization, should know better (and I believe based on personal communication with at least one of them that he does know better), yet the ISTQB crew persist in promoting a test-artifact-centric myth of testing professionalism; basically, a test factory myth. I believe they are doing this because it sells. It sells not because it works, not because it is in any way better than the context-driven and skill-centered approach to testing, but because to the uninitiated it is easy to understand. It works as a myth because its pseudo-science sounds plausible, but makes limited cognitive demands on managers.
As one example: “The various test procedures and automated test scripts are subsequently formed into a test execution schedule that defines the order in which the various test procedures, and possibly automated test scripts, are executed, when they are to be carried out and by whom.”
This sentence presumes to know the context of all test projects. The authors of the syllabus have no authority or reason to say that test procedures and scripts ARE or even SHOULD BE “formed into a test execution schedule.” Whoever wrote this sentence has confused a factory fever dream of his own with the testing reality of a great many test projects, including every test project I have ever been on in my 20 years as a tester and test manager. As a testing consultant who has taught in many countries and seen many companies, including many of the most famous software companies in the world, how can a practice that I have never seen even once be contemplated as a basic part of the craft? It’s absurd.
I can understand that somebody who is part of the ISTQB may obsessively pre-order his test artifacts. He may line them up like little butterflies in a perfect little collection, but there is no evidence that this is a desirable practice, nor a widespread one. How do rotten chestnuts like this get into your syllabus?
I believe that the ISTQB syllabus, which has many such examples, is a tool being used by unscrupulous or weak consultants to line their pockets by exploiting the fears of the ignorant. I return again to the fact that the syllabus is advertised without any mention of the salient fact that there is no consensus in testing about a “syllabus of testing”. What the ISTQB reports as a consensus is merely a consensus of their own faction. They know full well that many prominent and respectable thinkers in the field have rejected that way of thinking.
I think Francis Bacon was upset about a similar use of Aristotle’s work, which by the 16th century had become a tool to prevent learning, rather than foster it.]
Its not just with ISTQB but many of the (if not most of the)institutes,organisations,bodies or what ever you call are part of all this. This is my honest opinion being a test engineer/practitioner.
Unfortunately there has always been a wide gap between the theory and practice and it still exists to a very great extent.
The call of the day is for writings(including the academic syallabus) that are direct output of practice and ofcourse a justified and contextual application of the same knowledge.Testing activity as an entity has been seen as dependent activity(though it is so to certain extent), and tons of other sciences,logic,procedures,patterns,templates/artifacts got linked to it making it more compressed and limited in nature.
The answer for all the above is simple, follow the fundamentals,apply every thing as per the context and transform your self into a role demanded by the specific context.
For me testing has always been a transition from one state of mind to another (architect–>programmer–>DBA–>test engineer–>enduser)and iam sure most of us have seen this working.
Hey James. I was just rereading this, and wanted to mention that I got tired of seeing that definition of insanity quote alternately attributed to Einstein and Ben Franklin, so I tracked it down. In case you’re curious, here’s what I found:
“Insanity is doing the same thing, over and over again, but expecting different results.”
Source: Sudden Death, Bantam Books, New York, 1983, p. 68. http://en.wikiquote.org/wiki/Rita_Mae_Brown
It’s interesting to see Bacon saying something similar over 300 years earlier, and I like Bacon’s emphasis on trying new means…though I must say Brown’s phrasing is a bit more pithy.