This comes from an ISTQB advertisement they spammed me with, today:
“To ensure the quality of any software system, testers and QA professionals must thoroughly test the product. But how do you know that these tests are effective? If your team is conducting ad hoc, informal tests with little guidance or planning, the quality of the end product can be severely jeopardized—negatively affecting your bottom line.”
I don’t like to say things like this, nor am I comfortable supporting people who do. It’s not that it’s untrue– it is not necessarily untrue. But it is the kind of statement that fans the flames of a certain sort of Factory School bigotry in our industry. “Oh, you can’t trust testing unless it is pre-planned, pre-packaged, pre-approved, formalized, etc.”
Notice they say nothing about skill. It’s all about methodology, here, not skill. This kind of setup suggests that the next statement will be about the importance of factory-like test methodology. But that’s not what happens.
“The best way to be certain that you are providing customers with quality software is to make sure your team of testers is certified.”
Friends, I’m aware of no one in the industry– not even my worst enemies, not even Rex Black or Stuart Reid– who would publicly assert this or defend it. In fact, in debates against those of us who think certification is consumer fraud, the most typical move is for certificationists to say that certification isn’t even about skill, but rather about basic knowledge. “It’s a start” they say. “It’s a foundation.” (I reply that it’s a bad start and a bad foundation. Much worse than what it tries to replace.)
But then they allow this sort of advertising to go out! Completely undercutting their innocent-sounding plea! And they wonder why I complain that they don’t have the best interests of the testing craft at heart.
Notice that the “ad hoc and unplanned” stuff doesn’t even logically connect to certification. In fact, wouldn’t a highly skilled tester be far more likely to succeed with an ad hoc testing regime? When Roger Federer plays ad hoc tennis, I bet he still wins.
I think the reasons they start talking about methodology and end up talking about certification is A) their potential customers don’t understand the difference between skill and method, B) method is more concrete than skill, thus easier to evoke, and C) they know that what they say doesn’t have to be true or even logical, as long as it evokes horror and promises hope.
Oh, but there’s more…
“By taking the Software Tester Certification course and earning an internationally recognized certification in software testing, your team will gain the expertise needed to handle your greatest testing challenges; earn credibility and recognition as competent quality assurance professionals; and provide greater value to your organization.”
It’s internationally recognized? By whom? Some people who don’t study testing and some people who study testing and financially benefit from certification. Okay, but it is also internationally ridiculed by serious testers of many nations who wish to raise themselves to a level of skill that can’t be obtained in just a couple of days of training.
I recently encountered Dot Graham, now semi-retired, who told me that it hurts her feelings when people like me suggest that certificationists are only in it for the money. Dot is a sweet person. I don’t want to hurt her feelings. But I point her to advertising like this and I challenge her to explain it in any other terms. If not greed, then what, Dot? Stupidity? Pride?
Dot doesn’t want to argue with me about this. Of course she doesn’t. Rex Black doesn’t want to argue, either. Naturally. What answer could there be? Lois Koslowski once told me that “big dogs” don’t need to debate (in fairness, Lois Koslowski claims not to be a tester. I agree that she showed no testing competence or knowledge in the conversation we had. I just mention her because she did claim to be in charge of the ASTQB organization. Yikes!) This is capitalism in its ugly form– harvesting the ignorance and fear of others. Debate has no place here.
Is there no one in that self-declared professional community who reviews the advertising and stands up for professional temperance and humility?
James, I would be interested in seeing you finishing the statement “The best way to be certain that you are providing customers with quality software…”.
We had a team meeting today and one of the items that came up was providing quality software to the customer. Interestingly the discussion did not cover people needing to be certified.
[James’ Reply: There is no best way. That would be like a restaurant owner asking a chef consultant what is the best way to provide quality food.
But more to the point, why are you asking this question? You already run a restaurant. You must already have an idea about how to make software that works.
I’ll give you one hint. The most ESSENTIAL element in producing quality software is… having someone who knows how to program a computer. That’s really important.]
I was spammed with this as well and blogged about it early this afternoon on my site (http://josephours.blogspot.com/) [shameless plug]. Their stance, mission statement, and approach to the field is an affront to all testers, regardless of their ability. What is priceless is their mission statement that unequivocally states they support a “…qualification scheme…”. A scheme! They are not here to advance the craft, only to fatten their wallets.
I also received this email advert and I was shocked to read the statement.
“The best way to be certain that you are providing customers with quality software is to make sure your team of testers is certified.”
I didn’t bother to read on. There are many ways to insure your customers are receiving the best quality software. Certification is not one of them. I recently attended a talk on using metrics to justify testing resources. Typically people mentioned defect ratios. Not one person mentioned using the amount of testers who are certified as a way to justify the expense of testing. Such poor advertising will surely lead to the downfall of the ISTQB.
>>> To ensure the quality of any software system, testers and QA professionals must thoroughly test the product.
James, you have not addressed this… what is the deal about “thorough” testing ? how certified testers know that they have thoroughly tested? how does certification helps them to do so?
[James’ Reply: I assumed that “thoroughly” is a synonym for “adequately.” In other words, they are talking about good testing.]
>>>I think the reasons they start talking about methodology and end up talking about certification is A) their potential customers don’t understand the difference between skill and method, B) method is more concrete than skill, thus easier to evoke, and C) they know that what they say doesn’t have to be true or even logical, as long as it evokes horror and promises hope.
The last one is key — it creates a mass hysteria or horror among IT managers that their products are not tested well (by so called certified testers by following certified methods). The horror and panic that these certification folks creates mostly does the trick.
If not IT managers – newbies who are trying to make inroads into careers are surely lured into dirty tricks of the certification. Few, after getting certified, realize (if they are lucky read other stuff) that what is the real value of certification and some other get sucked deep into certification spiral, eventually end up in becoming stronger supporters.
Shrini
@James: Very well put. I got the email too and thought of you. The ISTQB guys really don’t give us a chance EVER to warm up to them. Just when you think about peaceful co-existence you get something like this and….
[James’ Reply: Well I think this is the comical unmasking of the truth behind their posturing. Early on in the certification movement it was all high-minded talk about professionalism. It was wrong-headed, even then, but they at least tried to answer the critics. But once they realized they could ignore the critics and sell to ignorant people directly, their tune became more arrogant. Now it’s just outrageous.]
@Darren: I don’t think THIS will be their downfall but it definitely looks as if the ISTQB has lost it’s lead somewhere. The clear Microsoft-stance (take over the world) seems gone. Now there’s not only the pure $$$ shining through but also a perceived disjointedness in their actions. Maybe this is the result of too many parties trying to get in on the game and the ISTQB-leads loosing their control. In the end that will cost them their chance of success. I think this is very similar of what happened to ISO 900x.
@Chris: “The best way to be certain that you are providing customers with quality software…”
Maybe if you do your release to UAT/Production and your customer is happy. I sure like my customers when they get back to me and say “job well done!”. That to me proves I delivered quality or at least to expected quality. That does not mean the software is released with 0 defects though. The customer is your most important stakeholder and what he/she perceives is what should guide you (I’m not saying the customer is always right either though).
Now you might say that’s an odd way of determining quality but I think experience is the No 1 way to improve testing skills above all else and experience can only be fully gained until a project is all done & dusted (and I take a working-inquisitive mind as a given).
Here are some figures from Germany from http://www.gulp.de, a portal for IT-freelancer (121543 inquiries for projects, 20918 offered projects)
(Know-how -> Markt & Trends -> GULP Trend Analyzer)
Prozentualer Anteil der gesuchten Qualifikation bezogen auf alle angebotenen Projekte eines Monats.
Detaillierte Erläuterung Anchor
Juni 2008 bis Mai 2009 ISTQB
Kontaktierte Freiberufler pro Projekt 6,9
Kontaktierende Firmen 30
Kontaktierte Freiberufler 241
Projektanfragen 354
Angebotene Projekte 51
Anteil angebotener Projekte in Prozent 0,2 %
If necessary, I can translate it.
[James’ reply: Yeah, what does this mean?]
Why should testers (or project managers or business analysts or database admins, or…) be “required” (or targeted) to have a certification, when the people writing the code (aka developers) aren’t?
So capitalism aside, why are non-developer roles ridiculed and downplayed and the target of such advertising? And to James’ point, why would we do this to ourselves? (Reenter capitalism).
My thoughts…
As far as I can tell these are some statistics for a job search from a german side. Here the rough translation of the searching statistics:
(Know-how -> Market & Trends -> GULP Trend Analyzer)
Percentage of searched qualitifcations related to all projects of a month.
Detail
June 2008 to May 2009 ISTQB
Contacted Freelancers per project: 6.9
Contacted companies: 30
Contacted Freelancers: 241
Project queries: 354
Provided projects: 51
Percentage of provided projects: 0.2%
I think the point is examplified to the international respected claim of the ISTQB. 0.2% of the jobs offered were based on the ceritifcation. Two out of 1000 jobs in Germany.
James,
As usual a very cogent blog post. I was going to write something to throw in some counterpoints, not to debate but offer alternatives. But it all comes down to, in my mind, the differences in knowing and doing.
Being educated (knowing) is good, but not being able to apply it (doing) is not good. Being educated and able to apply what you have learned in the proper context (ooo… snuck in the Contextual angle) is what is really needed.
The certification programs really don’t get that second part of the equation. We all need “basic foundation” knowledge to build off of. But how we apply it is the real key.
Certification can be a ‘tool’ of sorts. But as we all know “A fool with a tool is still a fool”. What that advertisement is is foolish and foolhardy.
[James’ Reply: I would agree with you, if the certificationists were even offering knowledge. But they aren’t. They offer mythology, buzzwords without substance, stuff they found in old Spam cans from the 70’s, confusion on a plate.]
I’m sorry, this is my favorite part out of the three quotes:
“…earn credibility and recognition as competent quality assurance professionals; and provide greater value to your organization.”
I’ve been testing software for five years now. But, even when I was only six months in, I had credibility and recognition amongst the people I needed it from…the people who sign my paycheck. The body of my work stood (and still stands) up to scrutiny. Project Managers, Directors and Vice Presidents listen to me and take my advice. My team and I provide value to my organization.
I’m actually insulted. I’ve read Caner, Bach (thank you), Weinberg, Bolton, and even Rex Black. I’ve put together a process incorporating ideas from each person, putting my own spin on things. Today, my team is able to work quickly and accurately on just about any task I give them. Minimal issues are discovered after production releases. Who are these people to try to tell me that they know better than me what will work here? That they can help me earn credibility, and provide greater value to my organization?
I must be doing something wrong. My record disagrees, but obviously I’ve messed up somewhere. Because I don’t have a piece of paper saying I know what I’m talking about.
[James’ Reply: Well said.]
I get what you are saying James, but who cares. Some people get that testing requires skill and you don’t get or prove skill through certification. Some people don’t. Those that don’t will fall into the trap and will be disappointed.
Why bother wasting any of your energy or those that get it on even talking about this. Times are changing, companies and the academic world are finally catching up, and the hope that good testers can be born from certifications lies in those companies that will fall behind.
[James’ Reply: I don’t think I’m wasting my time. I think a lot of people google about this stuff when they are new to the craft, and when they fail to see people like you speaking out against it, they assume that you support it.]
I have been paying attention and following the context driven thought for the better part of 09 and late 08. It seems there is a constant war or battle going on. Maybe that battle is more amongst the consultants versus those that work at the businesses. Either way it is not going to be won, through point out each others deficiencies. It will only be won by showing business value attained through projects utilizing context driven thinking. Everything else is just entertaining discussion.
Sometimes you need to let your son touch the thorn bush for him to understand that it will hurt.
[James’ Reply: I’m not limiting myself to one strategy, if that’s what you are worried about. But I do think this situation wouldn’t exist if a lot more people than just me would voice their opinion.]
It seems that I start learning from blogs like this. At least I’m trying, please be patient. Would I first try to argue that ISTQB might have some value, I learned from blogs like James, Michael, Pradeep, Cem, Matt that there is something more important. How your skills only can give value to organizations (tools and techniques only supporting your skills)
Of course there are also other “good” courses/workshops where you have to pay for information, only those seems to me to give you skills.
Related to ISTQB, if it is so good and we all should have it, then why not give it now for free. Provide us the information so we can teach ourselves, if we think we need more skills, then provide us those courses. I’m sure we are willing to pay for skills.
If it is so good, then share the knowledge openly like most blogs do. If it is so good let’s discuss and see if we can use it within certain situations.
Currently I don’t see any blogs or websites related to ISTQB were information is given for free with the intention to help us testers (with the exception of the ISQTB vocabulary (note: no quality statement of fit for usage from me). The only information I see that it should be good and needed and if you want more of it, pay for it first without any guarantee and promise that it does what it claims.
For ISTQB I thought it was a starting point, as everything needs a starting point, this doesn’t means that it is a bad one or a good one. That can only be told after gaining some skills. Gaining skills will make you adjust your route in the testing world. I believe as long as you are moving you are doing something and there might be a chance you do right things. For these you get the credits for the “bad” things you get the lessons and perhaps also skills. I don’t think this can be captured in an certificate.
If a certificate is the main goal then please make just one certificate then our goal is easily obtained. Currently there will be 3 level of certificates. This created the understanding that after “earning” them you have done it all. So there is nothing more to learn. If there is nothing more to learn, why is it needed? You keep only focused, am I doing it according to the method? if that is becoming the path, then I wonder how all statements about quality can be true as you train your brain to think like the method.
You might become a person walking with the face down to the ground checking if you are still on the track according the method. This will narrow your vision and not able to see what happens around you. I cannot imagine that people with a narrowed vision are able to deliver continues quality and that this should be the best way.
A few days ago I saw a tester CV where there was an ISTQB-certified logo in full colour & fat across the left upper corner. It was at least 3x3cm. It caused me to immediately flinch. I even asked myself if this person was working for the ISTQB. I wonder if ISTQB knew it’s logo was being used like that but if not they’d probably break out a bottle of Champagne now. This I figure would be the ultimate goal for them. Every tester CV has an ISTQB logo in the left upper corner or it will get shredded immediately!
The CV was excellent but why on earth would this experienced tester feel to show allegiance to something as trivial as ISTQB? Did he/she get paid to do this? An experienced tester should know better and if not….. I would have loved to interview the person but never got the chance to (was not for our unit). I was happy to hear though that he/she found work somewhere else. The last thing I need is someone in-house doing the ISTQB drum-roll.
But I must admit that CV shocked me. It showed a level of acceptance that I’m actively trying to prevent. Managers seeing such a good CV in association with the logo are bound to make a wrong connection. The only thing it really proved to me is that you can be a good tester in spite of having done ISTQB.
Hello Sir,
I really wish that 3 years ago when I came into this field I should have been guided by some one like you/Michael/Pradeep,etc… Because at the start of my career when I joined as a fresher I was asked by my TL to take up ISTQB. Out of curiosity & excitement I decided to opt for it but was really depressed by the notes and even coaching which was arranged for through the company. But now I make sure that who all newbies want to come into testing don’t fall for this trap.
Even today during my interviews/my friends/colleagues keep on asking me that why don’t I opt for ISTQB. What have I done in testing in the span of 3 years?? And my answer remains the same – I’m trying hard enough to sharpen my skills for being known as a “skilled tester” more than just a “certified tester”.
[James’ Reply: Thank you, Roshni!]
@James: Roshni has a point. Maybe we need a “Skilled Tester” badges.
@ Oliver,
Yes. Maybe then skilled testers will get noticed. In many companies some skilled testers keep getting rejected in the HR round itself just because of such certifications or that they haven’t done any testing diploma from any known institute. Many a times even in job sites we can see such examples:
Urgently required ISTQB/Diploma holder in S/w testing for a reputed MNC.
We can only hope & try from our side to at least minimize such things to make this field a better work place.
Since when have marketers ever sold an accurate picture of a product?
[James’ Reply: We’re not talking about small inaccuracies. It’s as if Honda said that its gasoline cars actually reduce global warming the more you drive them, or if they claimed their cars don’t need any fuel. These are big lies.]
On a more serious note, when I am running my own or other people’s training, the main thing I try to give students is a passion for testing, and an enthusiasm for learning. All good testers do this, Dot, Stuart, Paul G, Cem. Michael B, you, etc. The other thing is giving a context to what they are learning. Having trained testers who have floundered in their early careers without any knowledge, some sort of basic foundation knowledge is a useful starting point, but simplification of knowledge is a dangerous thing that an experienced trainer can point out, where someone less industry savvy may not.
[James’ Reply: As I’ve said many times, I don’t think the ISTQB provides a reasonable or useful foundation. I think the ISTQB view of testing is like creationist “science”: lot’s of mythology, uncritically mushed together.]
My favorite example is any foundation course that mentions coverage. Simplistic courses say go for 100% code coverage as a measure of great quality. I prefer not to mention coverage with novices, but when I do, I spend 5 minutes giving a history of coding practices, showing how simple 1960s Cobol like programs benefitted from code coverage, but the modern program of complex dataflows and objects with inherited properties may have 100% coverage ans still be unusable. The simplest example is the calculator tested with 100% statement and branch coverage that crashes when it tries to divide by zero.
Re standing up for professional temperance, late night after a day of a testing conference, zero temperance by most drinking testers, and some of them can barely even stand up either [grin]
I once trained a marketer in a certification (that included 3 hours of prac for every day of training). I explained that I wasn’t making him into a tester, but making him a more analytical thinker. He passed, and at least he sold the course more honestly after that…….
[James’ Reply: I teach coverage but I mean a different thing by it, probably. I don’t limit coverage to code. To cover is to examine with respect to a model of what can be examined.]
While I agree, I’m still putting “ISTQB and ISEB certified” on my resume, and if I can convince my manager to pay for the professional course I’ll do that too. An edge is an edge. Plenty of other professions have artificial and largely pointless barriers to entry – jumping over them shows commitment to the career. Your view of them largely depends on which side of them you happen to stand.
[James’ Reply: An edge is an edge. A lie is a lie. Manipulation is manipulation. Tautologies are tautologies. Yes, you are not required to make your public statements reflect your actual beliefs. In fact, I don’t recommend it, if integrity is a lower priority for you than making money.]
Hmmm mockery and a sideways ad hominem attack, classy. That’s a pretty lofty perch you’re pronouncing from chief. For the rest of us, there’s a bleedin’ recession on, and frankly putting food in my kids mouths is a lot more important than arguing the toss about whether some mickey-mouse training course is productive or not.
It gets me a foot in the door, nothing else. I still have to deliver. Me and my integrity sleep like a baby, thanks very much.
[James’ Reply: It’s not an ad hominem attack to point out the absurdity of the “edge is an edge” argument. It would be an ad hominem attack if I told you that you’re wrong because you are a poopy head. That’s not what I did. In fact, I would even say you’re not wrong, you simply made a plain and straightforward statement that you although you AGREE with me on the substance of my arguments, you CHOOSE to PRETEND to others as if you value something that you have implicitly claimed not to value. That’s called lying. You are telling us that you say things you don’t believe in order to get ahead.
Lots of people do that. Most people don’t brag about it.
You are doing this because you are afraid (you mentioned the recession) and because you believe it gets you work (I bet it does, among many employers). Integrity is a struggle. That’s one thing that makes integrity precious. That’s why those of us who hang onto it, despite a market that rewards us when we tell lies, get annoyed at those of you who’ve sold your integrity yet still claim to sleep soundly. That’s the sleep of the damned.
There’s hope for you, though. You can still take the ISTQB banner off of your CV unless you believe it really makes you a better tester.]
reading Colin Jackson’s comment, I would like to share the following idea.
As I am working in a very big semiconductors company which values the help for the community, I am involved in volunteering in giving lessons in a social project which gives IT training for youngsters in order that they will be able to start their careers in the hi-tec industry (here in Israel).
Few of the guys which just finished the QA course and are looking for jobs asked us wether we could deliver them the ITCQB material so they could get “certify” and hopefully increase their ability to get a job.
I went back to them with the following offer – I am qouting the mail I sent to their program co ordinator:
Following our conversation, here is my suggestion for a lesson for the QA course graduates.
Goal of the lesson is to suggest ways to create a portfolio for a beginner in testing. I am confident that gaining such experience will be valuable by potential employers.
Lesson content:
Give an overview of different options, like open source and community testing, and how to use them in order to gain a skill in testing.
How to be able to explain and demonstrate your experience.
Such lesson is just a kick off and suggesting ideas. the hard work will start when the lesson ends. No pain no gain 🙂
End of mail.
I still hope that they will accept this offer. I know that its harder then to prepre to a multiple choice memorizing exam. But the bottom line is that here are alternatives.
[James’ Reply: Wow, Issi, that sounds great.]
James, I think we have a similar view of coverage, from functions to data to a list of marketing claims on software packaging! The tradtionalist foundation courses I saw tended to only mention coverage in terms of lines of code, and hold 100% code coverage as some sort of gold seal of testing (which it is not as I explained above). A good instructor will use a broader definition, and explain that good coverage in one attribute may not imply any level of quality in another (and probably only talk of functional coverage etc and ignore code coverage with newbies)
I’ve seen this sort of thinking in a large corporation that I used to work for. They had an entire team that had bought into the Six Sigma cult (and by cult I mean that they were fanatically committed to it despite it’s results or lack thereof) and were constantly sending their team members to get their “Black Belt” certifications. The problem was that my team had to constantly clean up for them. They had difficulty completing their work on time and we often found major issues that they should have found. Management constantly came to us asking us to check their work or help them get finished on time. When we had to interact with them it was like talking to a brick wall; they couldn’t seem to get past their methodology to get to really skilled testing.
In my experience, it’s the same kind of thinking that demands large volumes of metrics. It gives management a warm, fuzzy feeling that things are going well in spite of reality. This team, and others that I observed in my time there, had beautiful charts and graphs that made it look they were really doing great testing. We would run some final checks on their area that was officially 100% executed/100% passing and within minutes find show-stopping bugs. I’m wondering if other people have had the same observations.
This is just one more reason why I have come to distrust certifications. When I interview people now and they start to tell me about all their certifications I always tell them that I won’t hold that against them, and then immediately start to ask questions that assess their skill level. Fortunately, I get very few of those types of candidates.
For me, certifications are like putting a seat belt on a motorcycle. It gives you a false sense of security, but when crunch (or crash) time comes it really doesn’t do anything for you.
Swallow this:-
The most unfortunate thing is, in the name of Customer feedback, Value certification push is being forced into the services at various Big organizations who claim to be the Big testing power houses. This to the effect that none of the certifications are testing related but ….no ones knows the mapping.
Astonishing thing is this is linked to their performance appraisal and even if they are the best of the lot, if they have not completed this ir-relevant certifications, they will be moved to lower rating and thus salary grade. Mind you, after 2 consecutive downgrades, you will be shown the door. How stupid or how employee friendly?
[James’ Reply: When doctors allow themselves to peddle fake remedies, unwitting people demand those, too. It’s up to us to police ourselves, and refuse to lie, even when it’s lucrative.]
James very curious question:
You have come hard on people prophesing on “Methodology” and not “Skills”, but to my surprise, on the front page, top left panel “Navigation” on https://www.satisfice.com, the first listing is “Testing – Methodology” and then “Testing – Tools”. Why is “Testing-Skills” missing on the whole page itself?
[James’ Reply: The short answer is because I don’t know how to package testing skills on a website… But also, I’m not opposed to methodology in any way. I’m opposed to idolization of methods. I have ideas about what to do and how to do it, just like everyone else. AND I invite you to be skeptical about what I offer, and rethink things for yourself.
I appreciate your question, BTW.]
Hi James,
As always, love your work. 🙂
Firstly, at a previous employer, we had a test manager declare a goal that he aimed to have all his test organization testing certified.
His test managers under him, weren’t so sure about this, myself included.
When asked by my manager (hesitantly) if I wanted the certification- I openly said “thanks but no thanks” and I explained that I’d been a tester a long time, and didn’t need to be ‘educated’ by people for 2 days, so that I could say I could test the way they think is best.
It is a con.
Its great for people who have failed at their chosen profession and want to be testers and want a quick way to add something “testerish” to their otherwise quality devoid resumes but little else.
As a tester, if you’re trying to find ways to cut stuff out of your resume, I think you’re in a good place and you don’t need such certification. If you need to pad your resume, then really you just need to work in the trenches, read, and work hard and learn. There is no magic bullet, and certification certainly isnt it.
For people who have already chosen testing as their profession- and take it seriously and think about what they do, why they do it, and how- ISTQB is a joke.
@Erik on Code Coverage.
At one place (which I cant name) circa 2003-5 we were encouraged to have high code coverage numbers.
I ended up hitting 95%.
Ill just say I proved a point.
It was very gameable.
I paired with my dev, cut out all debug code, all code that was not callable, removed all #ifdef’s and anything else that could possibly not be hit by the application being under test, then ran my automation against the instrumented binaries.
Just by doing those code removals pushed my code coverage from 67% to 95%.
In doing so, I single handedly drew into question the credibility of the process, and the product of that process- the numbers themselves.
What most people don’t understand about code coverage, is the difference between hitting blocks of code, and arcs of code.
Blocks as a code coverage metric is near worthless, unless you’re trying to hit code that is impossible to get to (see above refactor), or you’re flat out not even testing the functionality- more than likely it will get hit. The second it enters the block, its 100% covered.. useless…
Arcs on the other hand (eg. for,if,while,case:) add some value, as it least indicates that you’re trying different permutations.
Some value. Not an almighty measure of awesome, just shows you’re thorough, and can be a guide to cover holes in your testing that you missed- but the bigger problem of course is, if you were following TDD in the first place, there shouldn’t be any holes.
Seems a lot simpler to work smarter to begin with, than to work dumb then try and use code coverage to help you to fix it…
@Darren re:
>>Such poor advertising will surely lead to the downfall of the ISTQB.
You’d think that, but unfortunately, there are those of us that test software and inform customers- and allow them to make educated choices, and then there are those that use testing to blame others more effectively while seeking to avoid blame themselves. I’ve heard this kind referred to as “Teflon management” (nothing sticks). Fear, ISTQB sells well to the fearful, and will continue to for quite some time I can imagine. It seems like old school FUD with some curiously random spin.
James, that kind of literature makes me ashamed to have my ISTQB certification. I tasted throw up in my mouth when I read: “The best way to be certain that you are providing customers with quality software is to make sure your team of testers is certified.” It’s incredibly pedantic and audacious! It’s also just a flat out lie. No part of it is even a little true. Did they get the smoking lobbyists to work for them? (Wasn’t Thank You for Smoking a great movie? Harvey Dent, err…I mean Aaron Eckhart, was great.)
Speaking of smoking, if I decide to burn my certification, you’ll be the first to know. The video will be on youTube.
[James’ Reply: I will thank you for smoking your certification.]
Thanks for your blog.
I have been anti-certification since I started in technology — not just in my current QA days but also in my past days elsewhere in technology. For starters, they have always been maddeningly expensive. Even just the cost of taking the test themselves — never mind the cost of the prep classes (which are conveniently provided by the same people).
The necessity of the prep classes speaks to the nature of the certification. They are designed towards specific correct answers, rote memorization over innate capability — what education researchers call “near transfer”.
I took a Solaris sample certification test once. One question was “how do you determine the size of a file”. There was of course only one accepted answer, despite there being many different and valid ways to determine this in reality. I scoffed.
The problem is that now and again, particularly during the late 90s, employers demanded certifications before consideration. You’re required to have this and that certification, or never mind. I suppose the response argument would be “don’t work for that company”, and I’d tend to agree, but sometimes you need a job. As a result I’ve from time to time considered getting some sort of certification, despite the blatant theft of that industry, in order to add a little game-playing sparkle to the resume.
If we could convince the CTO/CIO/MIS/technology management of the world that certifications are meaningless and dependency on them is folly, I think that would be much more helpful than convincing technology staff of it.
[James’ Reply: I bet most of these employers who “demand” a certification aren’t really demanding it, but rather simply mentioning it. Of course, if you go through the HR group, they will hold to the rules because they don’t know any better. You need to network and get through to the hiring managers, directly.]
Hi,
I agree with your comments about ISTQB. It provides a glossary of terms and not much else. As a Tester for 25 years at the bleeding edge, I have run more testing engagements than I care to remember. I have 3 experiences with ISTQB: 1. Real testers laugh at it. 2. Those testers who demonstrate their capability by refering to their ISTQB qualification as proof of anything and parrot the ISTQB homilies generally can not find their bum to wipe it. 3. It is a real pain when I go looking for my next challange to see roles where ISTQB foundation certificate is essential. Essential for what? One of your commentators, obviously an experienced tester said that he has certification as it will help him get a job. It puts food in the mouth of his kids. Well, I can tell you it has zero nutrional value.
THe latest shit from a Software certification parasite:
With credentials behind your name, you will:
Start 2010 with the fundamental skills to complete tasks efficiently and address challenges
Stand out from your peers with your professional certification
Add value to your career path and provide greater credibility to your team
Become an elite member of the 115,000 certified testers around the globe
I’d rather stick a pencil in my eye.
[James’ Reply: Elite? Elite?! Hand me that pencil, I need to lose an eye, too.]