To be tested continuously, as term CI suggests, the code under the test must be running fast, failing fast, be as idempotent as possible, be easily configurable and produce adequate and quickly actionable output and logging
- If requirements like these are taken into account up-front, they change dramatically the way we think about code we release. They may sound like irrelevant concerns in the beginning of development cycle, but practice shows that at the end, software that satisfies such requirements is stronger, more resilient, and of higher quality. Also, it implies that we are “forced” to think about big picture when “simply coding” in our seemingly isolated environments
- If code that we release is to be tested continuously it must be “nice” to the database, file system and any limited or expensive resource that it may eventually hit. Also, it is very desirable that state of any “external” resource after the test will be left intact (not always possible, but at least desirable)
- Notifications of the test failures should be concise and well targeted – this why we should be concerned with efficient and precise error logging and reporting at all stages of development
- During Agile planning it is very useful to keep in mind Unit Testing. If scope of the Task makes Unit Testing effort impossible, it could be a sign of warning and prompt for an additional breakdown of a Task or a Story (it is not a rule that set in stone, but it is useful point to keep in mind)
- Thinking of Fast tests vs Slow tests, or Unit Tests vs Integration Tests and their effects on CI process helps to optimize this process, make the most of it, and in addition, helps to better understand the code base itself
Knowledge Transfer and Documentation by example
- One of the best ways to explain how system works internally is to show a fellow engineer a few running Unit Tests. Of course, tests could be too fine-grained to reveal the big picture. This why it is always good to have a few very straightforward, high-level tests that simply demonstrate the essence of the feature or selected piece of logic. We can think of them as Demo Test, or Spec Test. In a team environment and especially in any distributed team environment it could mean a lot
- Good Unit Test will self-document the code and can efficiently complement or illustrate any Javadoc entry. More importantly, “documentation” of this kind will naturally be kept up to date – otherwise build will break 🙂
- Good Unit Test can demonstrate not only a feature, but the idea or concept behind the feature
POC and R&D work plus some non standard applications
- When prototyping a new feature or when doing preliminary research we often end up writing standalone programs that are eventually become a throwaway code. Implementing these tasks as Unit Tests has certain merit – they become part of the code base (but not part of the production code line), they are easily executable as part of the standard testing framework, available to be revisited later, and discoverable by others for review and study (again, it does not apply to every situation, but it is something to think about)
- Many proprietary frameworks and 3rd party systems often feature certain internal or proprietary configuration parameters that frequently lack documentation. We often make use of such parameters, usually after considerable effort to tune them up for certain needs of our applications. But those parameters and their effect may change from release to release, or from environment to environment (for example, Oracle DB, its optimizer with its proprietary hints…, or only suggested JDBC fetch sizes used for batch processing). If effects and baselines of such “boundary” configurations are captured as part of automated Unit Testing, any unwanted change in behaviour could be caught proactively before it will reach Production environment. For example, Unit Test can automatically produce and capture explain plan for the query used in Java code, and assert against absence of Full Table Scans. This technique is also useful to detect changing data selectivity as it affects applicability of existing indexes.
Efficient bug fixing and troubleshooting
- When Unit Test coverage for some module or feature is sufficient enough, we can often completely avoid expensive cycles of deploying and running application in the container, requirements of database connectivity and other configurations, as well as need to attach to the running process with IDE or debugger of choice. Running a few unit tests (even after some modifications) to reproduce the scenario based on data collected from error logs or from user input, may be all that is required to locate the culprit
Happy Unit Testing!