Maturities models (TMMi, CMM, CMMi, etc.) are a dumb idea. They are evidence of immaturity in our craft. Insecure managers sometimes cling to them as they might a treasured baby blanket. In so doing, they retard their own growth and learning.
A client of mine recently asked for some arguments against the concept of maturity models in testing. My reply makes for a nice post, so here goes…
First, here are two articles attacking the general idea of “maturity” models:
Maturity Models Have it Backwards
Here is another article that attacks one of the central tenets of most maturity models, which is the idea that there are universal “best practices”:
And of course commercial tester certification, which I often ridicule on this blog, is a related phenomenon.
Here are some talking points:
1. I suggest this definition of maturity: “Maturity is the degree to which a system has realized its potential and adapted to its context.”
In support of this definition, consider these relevant snippets from the Oxford English Dictionary.
* Fullness or perfection of growth or development.
* Deliberateness of action; mature consideration, due deliberation.
* The state of being complete, perfect, or ready; fullness of development.
* The stage at which substantial growth no longer occurs.
2. I suggest this definition of maturity model: “a maturity model is plan for achieving maturity.”
By this definition, I know of nothing that is called a “maturity model” that actually is a maturity model. This is because maturity cannot be achieved by mimicking the “look” of mature organizations. Maturity is achieved through growing and learning as you encounter and deal with natural problems.
3. Maturity is not our goal in engineering. Our goal is to achieve success, satisfaction, security, and respect through the mechanism of doing good work.
No one gains success through maturity. It is not our goal. Some businesses benefit by the appearance of maturity, but that is a matter of marketing, not engineering. And regardless of how we achieve maturity, not all maturity is desirable. A creature approaching death is also mature.
Hey, blacksmithing is a mature craft, and yet look around… where are the blacksmiths? The world has moved on. We are in a period of growth, study, and creativity in testing. No one can say what the state of the art of our craft will be in 50 years. It will evolve, and we– our minds and experiences– are the engine of that evolution.
4. The behaviors of a healthy mature organization cannot be a template for success.
We achieve maturity by learning and growing as a testing organization, not by aiming at or emulating “mature” behaviors.
Maturity is a dependent variable. We don’t manipulate our maturity directly. We simply learn and grow, wherever that takes us. As any parent knows, you cannot speed up the maturation of your children by entreating them to “be mature.” Their very immaturity is partly the means by which they will mature. Immature creatures play and experiment. Research in rats, for instance, documents severe developmental problems in rats that were prevented from playing. Rats who don’t play as juveniles are much less able to deal with unexpected situations as adults. They cannot deal effectively with stress, compared to normal rats.
There can NEVER be one ultimate form or process that we declare to be mature, UNLESS our context never changes. This is because, in engineering, we are in the business of solving problems in context. However, our context changes regularly, because people change, technology changes, and because we are continuing to experiment and innovate.
Darwin’s theory of the origin of species is also a theory of the maturation and continual re-generation of species. As he understood, maturity is always a relative matter.
5. The “maturity model” of any outsider is essentially a propaganda tool. It is a marketing tool, not an engineering tool.
Every attempt to formalize testing constitutes a claim, on some person’s part, that he knows what testing should be, and that other people cannot be trusted to know this (otherwise, why not let them decide for themselves how to test?).
I have formalized testing, myself, and that’s exactly what I am thinking when I do so. But I do not impose my view of testing on any other organization or tester, unless they work for me. My formalizations are specific to my experiences and analysis of specific situations. I offer these ideas to my clients as input to a process of ongoing study and adaptation. To make any best practice claims would be irresponsible.
6. If you want to get better try this: create an environment where learning and innovation is encouraged; institutionalize mechanisms that promote this, such as internal training, peer conferences, pilot projects, and mentoring.
As my colleague Michael Bolton likes to say “no mature person, involved in a serious matter, lets any other mature person do their thinking for them.”
Mature people take responsibility for themselves. Therefore, don’t adopt anyone else’s “maturity model” of testing. Let your own people lead you.
Hi James,
Maturity models like OPM3 emphasize that maturity should be improved continuously through the stages of standardization, measurement, control and improvement.
[James’ Reply: Yeah, and I think that’s bullshit. Drop the maturity crap and just focus on learning and experimenting. Maturity is an uninteresting matter in and of itself, but the damaging thing about the maturity models I’ve seen is that they advocate standardizing BEFORE optimizing. This makes sense only if you started out knowing everything you need to know to do your job fantastically well. But we don’t. We have to experiment. We need to try radically different things.]
What we standardize, measure and control varies depending on the context and the options for improvement are based on that. Everyone’s situation is different and the maturity models just support to solve the problems.
[James’ Reply: Maturity models I’ve seen don’t support anything. I wouldn’t use them to wrap a fish.]
Too much of force/pressure to further one’s existing capabilities can be harmful. As you said, there should be freedom to experiment and learn on one’s own. At the end of the day, if the customer is happy we succeed no matter what the maturity level is.
I agree that the best practices are merely buzz words for marketing purposes and trying to impose them on people inhibits their growth.
Regards,
Aruna
http://www.technologyandleadership.com
Great James!
I went and looked up the TMMi documentation to have a quick review, but that won’t be happening soon as its 180 pages of reasonably solid text. Hmm…
Anyway, I recall learning a bit about this model a few years back and the disclaimer at the end of every blurb was something like “…but maintain a focus on continuous improvement”. This has always struck me as strange when considering level 5 (Optimization). The documentation mentions that to be at this level “an organization is capable of continually improving processes based on a quantative understanding of statistically controlled processes.” So if there is much to improve should they be at level 5? Or can they stay there because they are doing it with the understanding of statiscally controlled processes?
I’m sure there is much more to that particular argument, but I don’t have time currently to read 180 pages to find out. I’ll make it a challenge for myself.
A good read as always James. Cheers!
[James’ Reply: The people who created the model apparently don’t really understand statistical process control, since social systems are not amenable to statistical control, and software testing is a social system. Testing is an open ended search process. It is profoundly non-linear. No excellent test organization controls its processes statistically. So, wow, the marching band of stupidity has really outdone itself with that routine.]
I recently read the book “The Little TMMi: Objective-driven Test Process Development.” Yes, I am admitting this.
As I was reading through it, my thought was that although there were a number of good ideas for areas of improved efficiency, I was not quite sure why a company would go through the process of bringing in a TMMi expert and setting aside valuable resources to run a process improvement project so formally. It certainly was a huge investment for questionable payoff.
[James’ Reply: Name ONE good idea they offered.]
I would agree that we do not want to merely “emulate ‘mature’ behaviors.” If we are strictly emulating the behaviors, we cannot expect the same results as someone who has developed these behaviors through experience. For those who have developed these “behaviors” through experience, they are not so much behaviors as they are choices. I say choices because the actions are based on a depth of understanding. In other words, we need to learn and understand, not simply follow a script.
[James’ Reply: Yes, exactly.]
My Analogy:
A child could obviously not read Erik Erikson’s developmental stages to progress through them faster. I read about them in 9th grade psychology and I did not miraculously mature. However, a parent might read about Erik Erikson’s developmental stages to be more aware of issues with their children and to find better ways to deal with issues their child(ren) encounter at each stage. The developmental stages, in this example, serve as knowledge to help problem solve. I see the maturity model only as a source of ideas for improvement, a springboard so to say. One should not take everything in the book as the word of god. Some of the goals and practices may be helpful, some may not.
Anyways, there’s my defense for reading the book. 🙂
[James’ Reply: The difference between Erik Erikson and the models I complain about is that Erik Erikson did research, didn’t he? So did Piaget. Even so, there are several competing models of child development, and none that are universally acknowledged as supreme.]
Hi David,
Decisions have to be taken based on facts and statistics with the right people involved. We cannot solve problems without having a clear idea of the situation. I go by statistics that are facts/real data.
Statistics had helped me to drill down to the specifics when I had problems and has guided me on how to improve process. There was an organization-wide issue of late finding of bugs. I charted the bug trends for 3-4 releases to find the root cause. There were few “Bug Leakage” from one phase to another phase. Some critical high priority bugs that could have been caught in functional phase were filed late only during regression phase. This was due to a new feature owned by my QA team. I found the point of contact and spoke with her and came to know it requires feature knowledge transfer from one team to another team early before functional testing starts because her feature has impact on many teams. Even though the bug belonged to her feature it was caused by another team. She started thinking on these lines only after I showed her the bug trend statistics.
Regards,
Aruna
http://www.technologyandleadership.com
[James’ Reply: You don’t describe how statistics helped you find the “root cause.” But I bet you could have found this cause without statistics, too. In any case, the use of statistics has little to do with maturity or good practice, in an of itself.]
Dear James,
I can see different perspective coming from a well respected person like you in testing community. From my point of view there are two different points here “process maturity” and “people maturity”. No doubt the people involved in testing have to be innovative, explore and take the ground reality at each stage and act. As long as organization internally use this “Maturity Model” for test process improvement and don’t stop the innovation part coming from people ability to think, adapting the lessons learned by following the existing process and analyzing the results against goal, I don’t see any problem here.
[James’ Reply: Really? You don’t see a problem with ignorant bullshit? And you don’t see a problem with a fetish of progress based on the opposite of progress– premature standardization?]
I believe the process we may use definitely has stage of maturity (that is what model recommends) Managed, Defined, Measured and Optimized.
[James’ Reply: Yes, well, apparently you haven’t studied how people and organizations actually evolve. Look around you, man. Have you read any research about organizational development?]
from my experience organization process go through this stage but “Optimized” feeling is more of attitude problem than reality. But having a process acts like guideline so that consistency falls in place at organization, re-use helps in productivity improvement, measurement if defined correctly and captured help to assess where we stand against goal and act. But no where model state that stop innovating. This is my understanding.
[James’ Reply: Apparently your understanding completely contradicts mine. So what does that tell you? It ought to tell you that something strange is going on, at the very least. It ought to tell you that “experience” is not enough, since we both have experience. Only, I bet I’ve see more companies than you have in my 27 years in the business, including 12 as a traveling consultant.]
I do agree with you like there is no word called “best practice” which can be copied from organization to organization but we can definitely have a practice in an organization which improved over time and is working for that organization.
[James’ Reply: Only in context, and people are the most important part of that context.]
At the end I agree that the certification coming from the organization’s customer is the “process and people maturity level” reflection of the organization than anything else.
But I agree with your point 3, “Maturity is not our goal in engineering. Our goal is to achieve success, satisfaction, security, and respect through the mechanism of doing good work”
Best Regards,
Nagaraj
You mention insecurity and I find that very interesting. I think one of the reasons why our business is flooded with certifications and other “methods” such as CMMI is because people don’t know what other people are doing. So it’s fairly easy to come up with stuff and use them as a sales argument. The sad thing is that instead of talking about what we actually do, we then tend to talk about the sales stuff as if it was meaningful in any way. When will we realize that the emperor has no clothes, and what does it take to get there?
The challenge I find with maturity models is that they want to look at software development as an engineering discipline. There was a time in my career where I thought we were doing engineering, but now I see that we are not. When you build a bridge, you look at the banks, consider what load you need it to support, determine budget, and choose the kind of bridge to build that works with those parameters. Then it’s all about the mathematics and the construction.
[James’ Reply: If these people wanted maturity to be about engineering discipline, then I would expect them to show some shred of engineering discipline when engineering their maturity models. To engineer a maturity model is to build a tool that is intended to aid humans in a fundamentally human endeavor: getting better at their craft. This is not like engineering a bridge. At. All!]
In software dev, there may be 100 ways to solve a problem. We may have to discover solutions as we work. You need time to do that.
Yet as a test manager I want ways to reasonably predict outcomes, at least in terms of “when’s this going to be done?”. I am not just accountable for the work my team does, but also for delivering a release in more or less a given timeframe. Sure, I’ll push against it if my testers tell me they’re still finding serious issues past the point where we all guessed the software should have stabilized, or at least make stakeholders aware of the risks they’re taking on if they stick to the schedule. But unfortunately this is where maturity models are a siren song to management, in that they seem to offer the predictability that is desired.
I’d like to see some discussion on how to balance the craft of software development/testing with the needs of business so that both sides are adequately served.
[James’ Reply: I don’t see how anyone is served by applying models that don’t work.]
Hi James,
These processes are having lesser risk, hence companies don’t want to take risk in order to innovate somthing new. Tight time lines and disability of people management at spot of time causing this maturity level as helping hand.
[James’ Reply: They don’t have less risk. Behaving stupidly is not a lower risk methodology, dude. I didn’t invent the need to learn and grow. It’s just necessary. Do it or fail.]
Involving Experiments in processes are having risk hence no one want to take the initiative for change in processes.
[James’ Reply: Getting out of bed involves risk. But you did it, today, anyway. This is because there is more risk involved with refusing to get out of bed.]
Hence the view of maturity which you given is acutal maturity, it can not be defined on specific numbers or somthing which can be measurable. At end of the day first step is needed for change the game.
Warm Regards,
Abhay
In the current Agile environment I’m working in… I opened up a page to test, and a dead cat fell out.
Could you assess the maturity of the cat, Lou?
I liken the CMMI to Hersey-Blanchard SLT or the Shu Ha Ri concept for learning. Not every painter is an Il Furioso. Some first learn the basics, step by step and by rote. The challenge is when to kick them out of the nest to learn to fly. nuff meta4s
[James’ Reply: Most of my work, day in and day out, is training software testers. I don’t teach anything by rote. I don’t use “Shu Ha Ri.” On the day I meet each student, I kick him out of the nest. Experiencing the confusion of creatively solving a testing problem IS learning the basics.
Besides, the CMMI is a load of crap.]
Something interesting occured to me while reading your statement “This is because maturity cannot be achieved by mimicking the “look” of mature organizations.” The idea of mimicking mature organizations to try and be mature is similar to the behaviour of small children: they try to act like the adults they see around them, thinking it makes them more “grown up”, but it’s just a facsimile. Maturity cannot be faked – it’s a hard-won attribute that comes with time, effort, and both successes and failures.