Rapid Software Testing
When someone tosses you a program and says "you have one hour to test this" can you do it? Are you confident in your work? Can you explain what you did and why?
We offer this primarily as an onsite course, but sometimes we work with organizations that arrange to make it public. See our schedule for more information.
Contact James Bach for details.
This unique 3-day course introduces you to Rapid Software Testing, the skill of testing any software, any time, under any conditions, such that your work stands up to scrutiny. Based on the ideas of James Bach, Michael Bolton, and Cem Kaner, with substantial contributions by other members of the Context-Driven School of software testing, this is the closest thing in the business to a martial art of software testing. Because we emphasize exercises, it is challenging for experienced testers, but works for new testers, too. This course provides hands-on demonstrations and drills as well as portable heuristics that help you create tests quickly.
Rapid Software Testing
Developed and taught by James Bach, this 3-day, hands-on class introduces you to rapid software testing, a complete testing methodology designed for a world of barely sufficient resources, information, and time. Based on the principles in the book Lessons Learned in Software Testing: a Context-Driven Approach, this class presents an approach to testing that begins with personal skill development and extends to the ultimate mission of software testing: lighting the way of the project by evaluating the product.
The philosophy of rapid testing presented in this class is not like traditional approaches to testing, which ignore the thinking part of testing and instead advocate neverending paperwork. Products have become too complex for that, and testers are too expensive. Rapid testing uses a cyclic approach and heuristic methods to constantly re-optimize testing to fit the needs of your clients. Rapid testing isn't just testing with a sense of urgency, it's mission-focused testing that eliminates unnecessary work, assures that everything necessary gets done, and constantly asks what testing can do to speed the project as a whole.
One important tool of rapid testing we will cover is the discipline of exploratory testing-- essentially a testing martial art. Exploratory testing combines test design and test execution into one process that finds a lot of problems quickly. If you are an experienced tester, you'll find out how to articulate those intellectual processes of testing that you already practice intuitively. If you're a new tester, hands-on testing exercises help you gain critical experience.
If you outsource development or testing...
We have taught this class at outsource firms in India on behalf of their clients so that they can do a better job of testing without needing detailed test procedures. But more importantly, the rapid testing methodology is about getting a lot of value for the testing dollar (value that simply can't be reproduced by throwing untrained bodies at the problem) so that your top management won't see testing as a commodity activity that any stranger will do as well as you. Even if you outsource, you may want to have a core team of testers back at headquarters who can rapidly test products to check the "testing" done by outsource firms.
If you are burdened with clerical requirements...
We have taught this class in organizations pursuing the CMM and organizations subject to FDA and other regulatory requirements. Rapid testing is about thinking. As long as they want you to think well and find important problems quickly, this is a class that applies to you. However, we do advocate a lean form of test documentation, to the extent you can possibly lean it. We also teach session-based test management, which allows you to measure and document exploratory testing in a manner compatible with more "formal" process cultures.
Are you looking for a practical class?
In this class we test real software, under time pressure. You will practice cutting applications down to size with rapid idea generation techniques. You will practice critical reasoning on your feet, by yourself and in small teams.
James Bach originally developed this class from his own experiences at Apple Computer, Microsoft, Borland, and several software testing companies. This is the best of West Coast software testing practice. The methods presented are not hearsay, but represent what we do on a daily basis, on real test projects. The class is taught only by the people who own and author the material, so you are getting first hand knowledge.
Why rapid testing?
Most testing classes try to teach you how to test thoroughly. But their idea of teaching is to recite terminology and the names and descriptions of techniques. They don't build skill, and they don't help you break down and tame a realistically complex testing problem. Besides, almost none of us are given the time and resources to execute a "thorough" test process from beginning to end. Rapid testing is a way to scale thorough testing methods to fit an arbitrarily compressed schedule. Rapid testing doesn't mean "not thorough", it means "as thorough as is reasonable and required, given the constraints on your time." A good rapid tester is a skilled practitioner who can test productively under a wider variety of conditions than conventionally trained (or untrained) testers.
The other reason to study rapid testing is your career. Historically, testers have had trouble gaining the respect of developers and other people on a software project. After all, from the outside, the testing activity doesn't look like much. Most of the value of testing comes from how testers think, but even excellent testers struggle to articulate or justify their ideas about how to test. Rapid testing is a personal discipline, teachable and learnable, that allows you to think and talk about testing with confidence. By contrast, a conventionally trained tester generally is limited to complaining about how the requirements aren't fully documented, or about how some other condition necessary for arbitrarily thorough testing has not been met. That behavior rarely inspires respect.
Rapid testing is indispensable when you are asked to test something on short notice, off the top of your head, early in the development process, or when you're in the process of developing test cases and procedures for future use. This approach is also useful even when you're called upon to test more thoroughly, and given the time and resources to do so.
How does rapid testing work?
Instead of explicit algorithms and instructions, we emphasize skill development and heuristic methods.
A core skill is the ability to think critically. Thus, we discuss and practice the art of being skeptical and of separating observations from inferences. This is a thread that runs throughout the class. We will listen to you report bugs and challenge you to explain the relationship between your conjecture that something is amiss and the observations you made. A good tester thinks like a scientist or a detective.
Rapid test design is an organized process, driven by a set of concise heuristics (think of them as guidelines) designed to assure that you don't forget to test anything important. For new testers, the heuristics provide basic guidance. For experienced testers, the heuristics help you organize and access your experience, so that even under pressure, you perform like an expert and feel like one, too. With practice, you get better and better at testing rapidly while still being fully accountable for your work.
Another element we emphasize is exploratory testing, which is the opposite of pre-scripted testing. There are often good reasons to pre-script tests, but there are also many situations where defining and recording tests in advance of executing the would take far too long and find far too few bugs. In exploratory testing, the tester designs and executes tests at the same time.
Where did the rapid testing ideas come from?
From 1987 to 1995, James Bach worked mostly alone to develop a systematic heuristic-based test methodology that applied to commercial mass-market software projects. Traditional test methodology didn't work well for market-driven test project. Starting in 1995, he began to collaborate with other thinkers and writers in the field, who helped find and fix errors in his work, and helped extend it beyond the scope of market-driven projects. What began, for him, as his own vision of testing merged with other ideas to become a community vision. Some of his colleagues eventually came together to identify themselves as the Context-Driven School of software test methodology. Others helped create the Agile Alliance, which published the Agile Manifesto. Cem Kaner, James Bach, and Brett Pettichord published the first context-driven testing textbook Lessons Learned in Software Testing. James and his colleagues speak and write regularly about testing, doing their best to advance the state of the art.
The ideas in this class are drawn not only from experience, but are also grounded in epistemology, cognitive psychology, decision theory, and other fields. Testing is a far more interesting field than most people realize. We're at the crossroads of many other traditions.
The original motivation for all this was James' personal quest to be a truly expert software tester. It is an ongoing journey, and this class represents the best our community has to show for it, at any given moment. Our goal with the class is to propel each student forward on his or her own quest for expertise and self-confidence.
Who is the ideal student?
The ideal student is anyone who feels driven to be an excellent software tester or software test manager.
The class is useful to all levels of tester, but seems to be most appreciated by experienced testers who want to become expert testers. The class works well when strong-minded and skeptical students attend the class. They challenge the instructor and make the class better, just like testers should. We try to make the class the most stimulating intellectual experience you can handle.
Another ideal student is the tester whose job is to check the work done by offshore outsourcing firms. You don't have time to do a full-blown test project. So, learn how to make a brief test project work.
“James is the toughest teacher I have ever
seen and the better you do, the more he will push you. This is
because he wants you to learn...He has incredible skills to read
people and activate their minds. Getting an answer from him is
sometimes difficult, but if you are persistent enough, he will make
you answer your own question! Like I noted earlier, this training
gives you much more than you will ever imagine. You will understand
event reporting on a whole new level after the course.”
--Jari Laakso (Tester, Manager)
Also check out Bill Matthews blog post about taking the RST
"Surviving the Socratic Experience"
Also check out Bill Matthews blog post about taking the RST course; "Surviving the Socratic Experience"
Slideshow from a Rapid Software Testing class in Cambridge, England (March 2012) - courtesy of Rosie Sherry
In this course, you will learn:
- Concise, universal heuristics and models for instant test design
- How to tackle any product or product idea instantly
- How to analyze a test heuristic or practice
- How to test despite ambiguous or missing specifications
- How to deal with overwhelming complexity or confusion
- How to know when to stop or suspend the test process
- How to prepare and deliver an impromptu test report
- Rapid Testing Defined
- What is Testing? A Questioning Process
- The Themes of Rapid Testing
- How Rapid Testing compares with other kinds of testing
KEY IDEA: Rapid Testing is Personal
- The Test Project Context
- Testing Under Time Pressure
- The Importance of Testing Skill
KEY IDEA: Thinking Scientifically
- Mental Traps
- Questions, Explanations and Predictions
- Confronting Complexity
- Observation vs. Inference
- Models Link Observation and Inference
- Spotting What is Missing
- Using Heuristics
KEY IDEA: Know Your Oracles
- Consistency Heuristics
- Coping With Difficult Oracle Problems
- Quality Criteria and Oracles
KEY IDEA: Know Your Coverage
- Structural Coverage
- Functional Coverage
- Data Coverage
- Platform Coverage
- Operations Coverage
- Time Coverage
- Ask for testability!
KEY IDEA: Use Exploratory Testing
- ET is a Structured Process
- ET is a Cyclic Story-Building Process
- Testing to Learn vs. Testing to Search
- High Learning ET
- Contrasting Scripted and Exploratory Testing
- ET Dynamics
- Focusing and De-focusing Heuristics of ET
KEY IDEA: Know Your Procedures
- Four Elements of Test Procedure
- Focusing and De-focusing Heuristics of Test Procedures
- Exploiting Variation To Find More Bugs
KEY IDEA: Know Your Test Strategy
- One way to make a strategy
- General Test Techniques
- Value (or Risk) as a Simplifying Factor
- Cost as a Simplifying Factor
- Can and should tests be repeated?
KEY IDEA: Rapid Test Documentation
- The First Law of Documentation
- Common Problems with Test Documentation
- Concise Documentation Minimizes Waste
- Consider Automatic Logging
- Taking Notes
- Documenting Test Sessions
KEY IDEA: Rapid Test Reporting
- Reporting Considerations
- The Dashboard Concept
KEY IDEA: Getting started with Rapid Testing
Testing Exercises (distributed throughout the class)
- Test the Mysterious Sphere
- Wason Selection Task
- Test the Famous Triangle
- Test Cases for a Calendar
- Test This Dialog Box
- Find a Particular Bug
- Use Exploratory Modeling on a Small App
- Find an Oracle for Font Size
- Discover the Role of Repetition in Test Strategy
- Report the Completeness of Testing
- Exploratory Testing with Playing Cards
RST Class in Estonia 9/21/2011; Courtesy of Irina Ivanova
Comments from Students:
"I really enjoyed having the opportunity to attend this seminar at your company in "good old Virginia"! It was very valuable to me and has definitely provided me with new ideas...No doubt, this will be one of the memorable seminars. Big thanks also to onathan and Lenore for their organization and contributions. You guys are a great team!!!"
"Equipment, space, materials... all excellent."
"Content: very clear. The exercises really reinforced the concepts presented."
"I really like James' approach to teaching. I think learning is most effective when students are challenged to re-think assumptions. He openly invited comments, questions, challenges, which makes learning engaging and fun. I really like the way James handled students' comments and questions. Very impressive instructional skill."
"Excellent class. Being a new tester of only 3 months, I learned many ideas and techniques to improve my critical thinking and areas to explore."
"Class was structured well, good material, slick technology, fun diversions (video clips)."
"Now, will I be able to put the concepts to good use? I believe so, although it will require practice. After 15 years of highly structured development and testing, not to mention a lifetime of linear thinking, such "anarchy" makes me uncomfortable, and it will take me some time to get good at this style of testing. Thanks for presenting it as an alternative, and teaching us how to decide when to take advantage of it (instead of a more biased view)."
"I had taken James' Exploratory Testing class already, so this was somewhat of a review for me. However, the review impact was more effective this time around because I had more experience. The new material really solidified the basic principles for me. The facilities are excellent."
Excellent material on best testing practices-well seasoned from tried and true practices. This material is useful to develop individual test skills and would be effective to deliver to whole teams together. I am not exactly sure how I will successfully disseminate the wealth of information back to the organization when I return, but I will try my best. Very worthwhile -- an understatement. Thank you for challenging us and being generous with your experience and knowledge."
"You guys rock!! This class was a great experience! Loved the variety and guided hands-on exercises."
"A. Ideas I will consider with my team/company will be:
- Defining beforehand what our 'good enough' testing is
- Exploratory testing -- especially the note-taking portion
- Pairing of testers for complex work (at least a more conscious application of it to get the most)
- Claims testing and random testing
- Test planning
B. Good, stimulating, challenging presentation
C. Great venue
D. Learned a lot from James (and Jonathan)
E. It was fun too!"
"1. Great course, packed with useful tools, tips, approaches.
2. Good class space, hands on very good.
3. Wish we could have seen all the material discussed, but all said/did was useful.
4. Overall, best testing training I've had. Thanks!"
“A number of years ago, sometime in the early 2000s, I was invited to participate in a special three day Black Box Testing course. The venue was the Satisfice Test Lab, a converted theatre, in Front Royal Virginia. James Bach and Cem Kaner tried out new training material and exercises.
They used a cross teaching approach in which James tried to teach Cem’s stuff and Cem tried to teach James’ stuff. We did cool exercises, tied testing to lunar geology, made presentations and were constantly challenged to act purposefully, defend our positions and creatively think on our feet. The days were filled with insights, discovery and many great “aha moments”. The evenings were filled with friendly debate about what we were learning about software engineering, what we were learning about testing and most of all what we were learning about ourselves. The camaraderie, conferring and community have led me to a rich career and much continuing collaboration.
Some of my most important and memorable testing projects have been done with Jon Bach whom I met at the Black Box Testing course, Jon was helping James and Cem set up test environments, organize exercises, run activities and drive logistics. I learned a lot. It was a rich humbling experience. I have done a lot of fun work with our classmates and instructors. It was an energizing experience! “
Some highlights :
· Ross Collard christened me the “Reverend Rob” after a pitch I made as part of a class group exercise trying to convince James and Cem to use our clever approach to test Open Office.
· I was asked to test a “rubber ball” and put on the spot trying to learn “why?”