A key element for good quality software is a good automatic run test-suite, which contains both unit and integration tests. As Frederic P. Brooks already mentions in his book ‘The Mythical Man-Month’, developers (as other humans, too) are far away from being perfect. As software from the outside view often needs to be written 100% correctly (only one missing or wrong statement in the whole execution path can make a feature to be buggy), we need to have a reliable way to get quick feedback whether something works or has been broken. A powerful way to accomplish this is by checking the behaviour with automatic running test-cases. Though this practice is already state of the art, you often find systems with a weak test-suite. This article shows why the feedback-loop is so important to how improve your test suite by getting a testing culture and to introduce test-cases step by step.

Tests give you invaluable feedback

In following situations the feedback of tests is very helpful or even essential:

• Existing tests tell you that you didn’t break existing behaviour after doing adaptions. These regression tests show you a green bar if a code change did not affect working features.
• Tests help you to introduce new features. Writings new tests is done in parallel when touching production code. This way you can concentrate on this single feature and you further more get a quick response, whether the feature is implemented correctly (green bar) or not (red bar).
• Tests help you to reproduce a bug. You write a new test case which shows you a the bug as a red bar. With this you have a very good starting point to debug and fix it. When fixed you get a positive response (green bar shows up).

The dilemma of weak test-suites

Of course one single test-case won’t do the catch. Your aim should be have many test-cases which summed up represent your whole strong test suite. Unfortunately test suites are often weak, especially when changing code they don’t give you a red bar if something has been broken. The reason why a test suite is weak can be manifold:

No practice of testing, too few tests

The view of tests as central “development-drivers” is not practiced. Things are implemented and after that roughly tested in a manual way. There is no coding of automatic tests which prove that bug-fixes are done correctly or features work like specified. Testcode is seen as not-so-important stuff, because it does not show up in the production code and this way does not give real additional value to the customer.

Tests aren’t executed regularly

Tests which are part of the test suite exist, but their execution is not part of the build. Thus they are hardly executed and are triggered manually. Often this is the case because the tests need human intervention and cannot be run automatically by a build server or similar. It also can be that in fact you have a completely automatic test suite but is just not common that it is executed often. If tests aren’t executed regularly (e.g. once a day, after each commit), you lose the advantage of quick feedback. With this running the tests after a week/month or so, you don’t know what code change caused the test failures or errors. After a while developers will get reluctant about tests, they will be getting used to red bars and simply put will just ignore them at some point. In the end tests won’t be maintained and your test code gets more a burden than a help. Finally your test-code decays to dead code.

Missing or very expensive tools

For test automation the proper tools are very important. Nowadays you get a lot of good open-source test tools (Fit, xUnit-frameworks). Never the less for some special areas it is not trivial how to automate some testing steps, especially in integration testing (e.g. setting up and validating smart-cards contents, or testing message-oriented/asynchronous systems in general). Some vendors sometimes provide some more advanced testing tools but these are either very expensive or/and cannot be extended for special problems.

The dilemma solution

Above mentioned constraints to get a good test-suite are often hard but can be tackled. First of all everyone in the team needs to understand that tests aren’t given to the customer but are inherent important to the development process. Well, you don’t include you favorite IDE to the customer, never the less you couldn’t create software without it, could you? Testing continously needs to be a new habit, so give the team some more time to dig into this, like planning trainings for JUnit or test automation in general. If the team starts to value tests there still could be the problem that there are hardly any automated tests, which can be very demotivating (‘The where to start…?’ issue). Never the less you can start up by writing tests for all new features you want to implement or for areas you want to refactor. When knowing a bug always reproduce a bug with a test case initially. After doing this for a while you will see how your backing safety-net will grow bigger and bigger. Of course the test cases of the test division won’t be replaced by newly introduced test cases of the developers. There still are many tests which are difficult or cannot be automated at all (like user acceptance or GUI tests). So see developer-testing as supplements and not replacements for the “traditional” tests. Of course the testers could also help to improve the automated integration tests, because they are more black box focused. In a very mature test focused process a tester for instance would attach a reproducing integration test to a bug report.

As mentioned not executed tests some day will end up as dead code. To avoid this introduce certain triggers (e.g. nightly, after each commit) to run your test suite. See the test runnings as part of the software build itself. This view is very powerful and is also recommended by the Continous Integration concept. If run very regularly developers will soon start to appreciate the tests and will have “fun” with extending test-suite. Looking at the visible green bar or getting rid of a red one can be very motivating.

In former times test tools often had been something more for the people in the test division. The picture has changed a lot. Automated tests created directly from the developers is state of the art and is supported by many useful and open-source tools (Fitness, xUnit, db-unit, Mock-Frameworks, soap-ui etc.). They are extensible and can be combined to get an even more powerful testing tool. Further more Continous Integration servers for running and triggering tests are mature, free and can be extended (CruiseControl, Continuum, Hudson etc.).

Summary

Generally the view of tests from the developers side has changed a lot. Especially since the rise of Unit-testing, some new integration test-tools and the concept of Continous Integration, test-creation by developers is much more common. Even if your organization has not introduced this practice you can do so by building up your test-suite incrementally and executing it regularly. The harder part will be to get the developers to a testing routine/culture but this effort will far be outweighed by the benefits.