Sunday, 21 June 2009

Tests Are Useful Before They Are Run.

Many engineers, including myself, thought that testing was a means of proving a solution did what it was supposed to do. The customer would present you with a set of requirements, you would produce a solution and prove it as such by running tests.

If you were more forward thinking than me, you might even think that it is useful in the development process. Refactoring code is a very good way to break a system, testing proves that your changes have not affected functionality.

Experience is showing that this was a very innocent view of testing.

The process of generating test cases is just as useful as the finished tests themselves.

We are finding that generating acceptance tests highlights conflicts and questions for the architects very early in the development process. This means that uncertainties can be addressed before resources are used on requirements which require refining. Not only does this save our team resources (and sanity), it also helps other teams. Numerous times we've found problems at this stage which have have effects on other teams.

This process also helps the customer. The customer is not (generally speaking) technically minded and whilst they might be able to tell you what a system should do (happy path) , they will probably not be able to tell you everything that is shouldn't do or how it should handle events outside normal operation. It takes a technical mind to spot the holes in a requirements document and it is often that these are highlighted in the test case generation.

Given these (rather superficial) observations, it seems that testing fulfills the following development needs:

  1. Highlights architectural problems before resources are spent.
  2. Aids in proving that a system does what the customer asked for.
  3. Ensures that a system's functionality hasn't changed after refactoring.
  4. Provides useful information to aid debugging.
  5. Can be a useful tool for finalising requirements from the customer.

No comments: