As I've been preparing to embark on my journey to build an automated testing framework around Huddle, I've come across some very interesting articles about the argument between manual and automated testing strategies.
It's safe to say that the general consensus found in the articles, blogs, and white-papers is that automation cannot supplant a solid manual testing approach. Of course, it can greatly
assist the overall
QA process in a number of ways.
Here's an interesting quote about automated tests from the article "
Test Automation Snake Oil" by
James Bach:
Assure that the product is mature enough so
that maintenance costs from constantly
changing tests don't overwhelm any benefits
provided.
Huddle is indeed a relatively immature product. Nevertheless, I feel that the tools available to us (
TestComplete) and the framework models (Logical
Fuctional, Physical Object, Composite Execution Behaviors, and Decoupled Verification) will allow us to get our feet firmly on the ground with a basic automation suite. I really enjoyed reading Michael Hunter's "
Automation Stack" articles and learning about these incredibly powerful, yet simple, models.
In another interview, Bach had this to say about writing overly detailed tests:
Since written test procedures cost so much and have so little value as instruments of bug finding, it surprises me that almost the entire testing industry accepts without question that most tests should be written down in procedural form.
Bach is a huge proponent of what's called "Exploratory Testing" which is basically a glorified name for: "Let-Smart-People-Play-With-And-Try-To-Break-Our-System Testing". In my eyes, an optimal testing strategy combines elements of automation (e.g. perform setup, check easily verifiable system states) with elements of manual testing.
In my opinion, loosely scripted test cases can guide testers into highly-used and error-prone areas of the system. Once there, the tester can use his or her intuition to develop new test cases and, most likely, uncover bugs.
I do agree that creating overly detailed and
completely unambiguous tests can have the effect of putting "blinders" on a tester's head. Testers should always be critical of scripted test cases, results from automation, and the testing process in general. At Agile, we have the luxury of learning best practices and improving on our processes each day.
-KRM
Labels: automation, bach, manual, QA, taso, testing
