Rapid Software Testing: A walk on the wild side
James Bach recently ran a Rapid Software Testing (RST) workshop at bol.com and I was fortunate enough to participate in this engaging and informative three-day event. Bach is regarded by many as one of the thought leaders who reshaped the testing industry within the modern software development landscape (alongside Michael Bolton).
This workshop isn't like any other testing training courses I've ever taken. It offers a whole new experience I find to be of a more cognitive nature. It doesn't just focus on certifying testers by conveying most well-known industry software testing standards like ISO 29199, ISTQB, Tmap Next or ISEB (now defunct).
Throughout the workshop, we were challenged to take on different assignments like testing tasks, hot seats (putting some of us in the spotlight) and even a couple of magic tricks. These illustrated how testers are sometimes misguided or even blindsided when testing products against hard requirements. Even more so in situations where there are no exact requirements (have 100% prefect requirements ever even existed and stayed that way?)
The cornerstones of Rapid Software Testing primarily revolve around a couple of instinctive notions, namely Heuristic, Context, Model, and Oracle. I will elaborate on each of these using a series of examples.
Heuristic
In my book, “Heuristic” is a word that describes an approach to solving problems. At the beginning of the workshop, James put all our heuristics to the test. Pulling up a Word Processor program, he asked us to test it. In fact, his exact words were: “How will you test this? What’s your test procedure?” Some of us were dumbfounded, others overwhelmed. After all, it’s a Word Processor, how hard can it be?
Now, in retrospect I think some or maybe even all of us were too used to the idea of getting all the information prior to any testing. Because we thought we had all the information we needed. And then it sunk in. More often than not, we don't think to question the obvious.
For example, it took us a while to come with questions like: “What do you mean by procedure?”, “What exactly it is that you want us to test?” (mind you, being a certified tester myself – the first thing that came to mind was test cases as soon as I heard the vocabulary “test procedure”, at least that’s what ISTQB/IEEE 829 calls it). Which brings us to the next subject: Context.
Context
We all know what context is, right? Let’s find out by going back to the Word Processor exercise. Some of us were stunned for a minute or two as we were trying to figure out what to ask, what to do next, what sort of resources we had access to.
A minute later questions quickly started coming out. With Bach constantly probing and challenging us intellectually, we eventually realized we were practicing what Rapid Software Testing advocates.
We were trying to put some context into the assignment. Without such context, we had no starting point, no plan, potentially obscured results, not much of an endgame. Ergo, to drive effectiveness, it is pertinent to make your testing effort context-driven.
Model
I’ll thread carefully on the subject of models but I would like to put test modeling in perspective.
Let’s compare it to testability review activity in a conventional test approach whose purpose is to establish the testability of the test basis. Simply put: we want to create a mental map of a product’s expected behavior by visualizing how it will react to various aspects around it.
By collecting these aspects and studying the relations between them, we can recognize and anticipate what the product should or shouldn’t do in any given situation. It’s a deterministic effort to predict the product’s behavior. Models don’t have to be explicit, they can be mental models as well.
Oracle
I think the easiest analogy to oracles is the “A-ha!” moment we all have once in a while. After all, an oracle within computer science is regarded as a mechanism to observe if something has passed or failed. Oracles are used to reason and to anticipate the outcome of the testing effort.
Allow me to quote some expressions from the workshop to illustrate the use of oracles: “I have reasons to believe that this functionality is correct because I’ve run series of tests for determinism and the stakeholders concurred with my findings.” or “It is within reason that the test failed because the data was intentionally corrupted.”
The exception proves the rule
The $ 64,000 question you might be asking is: “Does that mean ISTQB or any other software testing certifications have no place or hold no value in the modern-day testing realm?”
Well, that depends on the kind of products you’re testing or the nature of the business you’re in. For example, N-wise testing is commonly practiced in very specific domains, like the medical industry, that call on formal test techniques. Or state transition testing is of great use in occupational safety solutions. In addition, these industries may have legal inferences, bounded by ISO rules. Therefore, it makes perfect sense for test engineers to follow a more protocol-based route to testing.
Conclusion
As a test professional that earned most of the common testing certifications, I find Rapid Software Testing to be the most practical and the easiest method to incorporate into my current day-to-day testing efforts.
Primarily because Rapid Software Testing isn’t restricted to best practices or patternistic rules, but instead relies on critical thinking and common sense. In addition, RST tends to lend itself very well in situations where software specifications are less prevalent – a situation commonly occurring in Agile projects with short delivery intervals. In most cases, this type of situation calls for focus on constant communication, for adapting to changes and spontaneity rather than following best practices, templating or social dogma.
All in all, the main takeaway from the workshop is: be brave, be the bad cop, embrace spontaneity, use your intuition, avoid over-strategizing, and remember that the obvious answer is also the right one.