Test Methodology

This is an unordered repository of a few of the test methodology documents that exemplify our approach to testing.

Most documents on this page are in PDF format. Download Acrobat Reader to view them.

Internet Explorer users: We've discovered that the IE Acrobat plug-in sometimes doesn't properly display the contents of PDF files. If you experience trouble, right click on the link and select "Save Target As..." Once you save the PDF file to your local drive, you can open it up without using IE and it should display properly.

Heuristics of Testability

This is a list of ideas for what makes a product more testable. It can help testers and developers improve the testability of a product so that testing goes faster and takes less effort.

Session-Based Test Management

Exploratory testing (sometimes referred to as "ad hoc" testing) is a creative, intuitive process. Everything testers do is optimized to find bugs fast, so plans often change as testers learn more about the product and its weaknesses. Session-based test management is one method to organize and direct exploratory testing. It allows us to provide meaningful reports to management while preserving the creativity that makes exploratory testing work. This page includes an explanation of the method as well as sample session reports, and a tool we developed that produces metrics from those reports.

SQA for New Technology Projects

When you're doing a 1.0 product, you can't rely much on experience to guide your SQA process. You also don't have regression test suites or any other specialized test materials that you can reuse. Meanwhile, the product itself is probably changing at a high rate. It's poorly documented and you may not be the first to know about major changes.

This document is a set of ideas for dealing with that situation. It begins with the idea that you have to change your thinking from a task orientation to a risk orientation.

General Functionality and Stability Test Procedure

(for Microsoft Windows 2000 Application Certification)

I produced this procedure for Microsoft to help them do a better job of assuring that applications that claim to be Windows 2000 compatible really are compatible. The procedure itself is documented in 6 pages. As far as I know it is the first published exploratory testing procedure. It's used along with a second non-exploratory procedure (which is 400 pages long!) to perform the certification test. What's interesting about that is the fact that my 6 pages represent about one third of the total test effort.

Satisfice Test Context Model

This is version 1.0 of a model I use to help me analyze software test projects. It depicts the major elements in the context of testing that should influence choices about test strategy, test logistics, and testing products.

Satisfice Heuristic Test Strategy Model

This is a model I've been using for years. It used to be called the Tripos model. I use this model to organize my thoughts about all the elements of test design. By memorizing the various checklists, I am able to rapidly generate lots of ideas for how to anything. This is a classic example of a set of testing heuristics.

Test Plan Evaluation Model

This is a model I use when I'm reviewing and critiquing a test plan. It lays out what are, in my opinion, all the interesting issues to consider concerning a test plan and associated documents. One of the interesting features of this model is a set of test project heuristics.

Test Plan Building Process

This is an experimental process for evolving a good test plan. I'm still experimenting with it. It's an example of a "forward-backward" process, where you proceed concurrently on all tasks, rather than linearly through each task in a predefined order. It's also yet another example of a heuristic approach to testing. This procedure doesn't tell you what to do, so much as suggest what to think about.

Satisfice Inc.