This is an extract from the Unit Testing Succinctly eBook, by Marc Clifton, kindly provided by Syncfusion.

The previous articles have touched upon a variety of concerns and benefits of unit testing. This article is a more formalized look at the cost and benefits of unit testing.

Unit Test Code vs. Code Being Tested

Your unit test code is a separate entity from the code being tested, yet it shares many of the same issues required by your production code:

• Planning
• Development
• Testing (yes, unit tests must be tested)

In addition, unit tests may also:

• Have a larger code base than the production code.
• Need to be synchronized when production code changes.
• Tend to enforce architectural directions and implementation patterns.

Unit Test Code Base May Be Larger Than Production Code

When determining whether the tests can be written against a single method, one should consider:

• Does it validate the contract?
• Does the computation work correctly?
• Is the internal state of the object set correctly?
• Does it return the object to a “sane” state if an exception occurs?
• Are all code paths tested?
• What setup or teardown requirements does the method have?

One should realize that the line count of code to test even a simple method could be considerably larger than the line count of the method itself.

Maintaining Unit Tests

Changing the production code can often invalidate unit tests. Code changes roughly fall into two categories:

• New code or changes to existing code that enhance the user experience.
• Significant restructuring to support requirements that the existing architecture does not support.

The former usually carries little or no maintenance requirements on existing unit tests. The latter, however, often requires considerable rework of unit tests, depending on the complexity of the change:

• Refactoring concrete class parameters to interfaces or abstract classes.
• Refactoring the class hierarchy.
• Replacing a third-party technology with another.
• Refactoring the code to be asynchronous or support tasks.
• Others:
• • Example: Changing from a concrete database class such as SqlConnection to IDbConnection, so that the code supports different databases and requires reworking the unit tests that call methods that were dependent upon concrete classes for their parameters.
  • Example: Modifying a model to utilize a standard serialization format, such as XML, rather than a custom serialization methodology.
  • Example: Changing from an in-house ORM to a third-party ORM such as Entity Framework may require considerable changes to the setup or teardowns of unit tests.

Does Unit Testing Enforce an Architecture Paradigm?

As mentioned previously, unit testing, especially in a test-driven process, enforces certain minimal architecture and implementation paradigms. To further support the ease of setting up or tearing down of some areas of the code, unit testing may also benefit from more complex architecture considerations, such as inversion of control.

Unit Test Performance

At a minimum, most classes should facilitate mocking of any object. This can significantly improve the performance of the tests—for example, testing a method that performs foreign key integrity checks (rather than relying on the database to report errors later) shouldn’t require a complex setup or teardown of the test scenario in the database itself. Furthermore, it shouldn’t require the method to actually query the database. These are all performance hits to the test and add dependencies on a live, authenticated connection with the database, and therefore may not handle another workstation running exactly the same test at the same time. Instead, by mocking the database connection, the unit test can easily set up the scenario in memory and pass the connection object as an interface.

However, simply mocking a class is not necessarily the best practice either—it might be better to refactor the code so that all the information the method needs is obtained separately, separating out the acquisition of the data from the computation of the data. Now, the computation can be performed without mocking the object that is responsible for acquiring the data, which further simplifies the test setup.

Mitigating Costs

There are a couple of cost-mitigating strategies that should be considered.

Correct Inputs

The most effective way of reducing the cost of unit testing is to avoid having to write the test. While this sounds obvious, how is this achieved? The answer is to ensure that the data being passed to the method is correct—in other words—correct input, correct output (the converse of “garbage in, garbage out”). Yes, you probably still want to test the computation itself, but if you can guarantee that the contract is met by the caller, there is no particular need to test the method to see if it handles incorrect parameters (violations of the contract).

This is a little bit of a slippery slope because you have no idea how the method might be called in the future—in fact, you may want the method to still validate its contract, but in the context in which it is currently used, if you can guarantee that the contract is always met, then there is no real point in writing tests against the contract.

How do you ensure correct inputs? For values that come from a user interface, appropriately filtering and controlling the user’s interaction to pre-filter the values is one approach. A more sophisticated approach is to define specialized types rather than relying on general purpose types. Consider the Divide method described earlier:

public static int Divide(int numerator, int denominator)
{
  if (denominator == 0)
  {
    throw new ArgumentOutOfRangeException("Denominator cannot be 0.");
  }

  return numerator / denominator;
}

If the denominator was a specialized type that guaranteed a non-zero value:

public class NonZeroDouble
{
  protected int val;

  public int Value
  {
    get { return val; }
    set
    {
      if (value == 0)
      {
        throw new ArgumentOutOfRangeException("Value cannot be 0.");
      }

      val = value;
    }
  }
}

the Divide method would never need to test for this case:

/// <summary>
/// An example of using type specificity to avoid a contract test.
/// </summary>
public static int Divide(int numerator, NonZeroDouble denominator)
{
  return numerator / denominator.Value;
}

When one considers that this improves the type specificity of the application and establishes (hopefully) reusable types, one realizes how this avoids having to write a slew of unit tests because code often utilizes types that are too general.

Avoiding Third-Party Exceptions

Ask yourself—should my method be responsible for handling exceptions from third parties, such as web services, databases, network connections, etc.? It can be argued that the answer is “no.” Granted, this requires some further up-front work—the third-party (or even framework) API needs a wrapper that handles the exception and an architecture in which the internal state of the application can be rolled back when an exception occurs and should probably be implemented. However, these are probably worthwhile improvements to the application anyway.

Avoid Writing the Same Tests for Each Method

The earlier examples—correct inputs, specialized type systems, avoiding third-party exceptions—all push the problem to more general purpose and possibly reusable code. This helps to avoid writing the same or similar contract validation, exception-handling unit tests, and allows you to focus instead on tests that validate what the method should be doing under normal conditions, that being the computation itself.

Cost Benefits

As mentioned previously, there are definite cost benefits to unit testing.

Coding to the Requirement

One of the obvious benefits is the process in formalizing the internal code requirements from external usability/process requirements. As one goes through this exercise, direction with the overall architecture is typically a side benefit. More concretely, developing a suite of tests that a specific requirement is met from the unit perspective (rather than the integration test perspective) is objective proof that the code implements the requirement.

Reduces Downstream Errors

Regression testing is another (often measurable) benefit. As the code base grows, verifying that existing code still works as intended saves considerable manual testing time and avoids the “oops, we didn’t test for that” scenario. Furthermore, when an error is reported, it can be corrected immediately, often saving other members of the team the considerable headache of wondering why something that they were relying on is suddenly not working correctly.

Test Cases Provide a Form of Documentation

Unit tests verify not just that a method handles itself correctly when given bad inputs or third-party exceptions (as described earlier, try to reduce these kinds of tests), but also how the method is expected to behave under normal conditions. This provides valuable documentation to developers, especially new team members—via the unit test they can easily glean the setup requirements and the use cases. If your project undergoes a significant architectural refactoring, the new unit tests can be used to guide developers in reworking their dependent code.

Enforcing an Architecture Paradigm Improves the Architecture

As described earlier, a more robust architecture through the use of interfaces, inversion of control, specialized types, etc.—all of which facilitate unit testing—also improve the robustness of the application. Requirements change, even during development, and a well-thought-out architecture can handle those changes considerably better than an application that has no or little architectural consideration.

Junior Programmers

Rather than handing a junior programmer a high-level requirement to be implemented at the skill level of the programmer, you can instead guarantee a higher level of code and success (and provide a teaching experience) by having the junior programmer code the implementation against the test rather than the requirement. This eliminates a lot of bad practices or guesswork that a junior programmer ends up implementing (we’ve all been there) and reduces the rework a more senior developer needs to do in the future.

Code Reviews

There are several kinds of code reviews. Unit tests can reduce the amount of time spent reviewing code for architectural issues because they tend to enforce architecture. Furthermore, unit tests validate the computation and can also be used to validate all code paths for a given method. This makes code reviews almost unnecessary—the unit test becomes a self-review of the code.

Converting Requirements to Tests

An interesting side effect of converting external usability or process requirements to formalized code tests (and their supporting architecture) is that:

• Problems with the requirements are often discovered.
• Architectural requirements are brought to light.
• Assumptions and other gaps in the requirements are identified.

These discoveries, as a result of the unit test process, identify issues earlier in the development process, which usually helps to reduce confusion, rework, and therefore, reduces cost.