Lack of Will

A core problem with quality in our industry is lack of will.

Lack of “will work”, that is. This is because it’s much easier to tell that a product can work than that it will work. And too often it turns out that products will not work in some situations even though they can in others.

Yet, many testers, developers, and managers are recklessly confident in the will part when they’ve only observed the can part.

I often hear someone say that their smoke test suite “just checks that the basically functionality works.” But even this modest sounding goal is impossible to achieve. You can’t derive will from can, unless you give up certainty (“it will work, and I might be wrong”), or you run every possible test (and you can’t do that).

So, the claim of “…it works” is shorthand for something more uncertain, like this:

“During the tests I performed, I looked for cases where the product did not sufficiently fulfill the requirements I was testing for, but I did not see any. Furthermore, I have performed enough of the right kind of tests to justify confidence that the product probably will fulfill those requirements in the future for other people in other cases.”

Or more simply:

“It appeared to meet [some requirement] to [some degree] while I was testing it. It’s possible that the product works.”