This is an extract from the Unit Testing Succinctly eBook, by Marc Clifton, kindly provided by Syncfusion.

The phrase "proving correctness" is normally used in the context of the veracity of a computation, but with regard to unit testing, proving correctness actually has three broad categories, only the second of which relates to computations themselves:

• Verifying that inputs to a computation are correct (method contract).
• Verifying that a method call results in the desired computational result (called the computational aspect), broken down into four typical processes:
• • Data transformation
  • Data reduction
  • State change
  • State correctness
• External error handling and recovery.

There are many aspects of an application in which unit testing usually cannot be applied to proving correctness. These include most user interface features such as layout and usability. In many cases, unit testing is not the appropriate technology for testing requirements and application behavior regarding performance, load, and so forth.

How Unit Tests Prove Correctness

Proving correctness involves:

• Verifying the contract.
• Verifying computational results.
• Verifying data transformation results.
• Verifying external errors are handled correctly.

Let’s look at some examples of each of these categories, their strengths, weaknesses, and problems that we might encounter with our code.

Prove Contract is Implemented

The most basic form of unit testing is to verify that the developer has written a method that clearly states the “contract” between the caller and the method being called. This usually takes the form of verifying that bad inputs to a method result in an exception being thrown. For example, a "divide by” method might throw an ArgumentOutOfRangeException if the denominator is 0:

public static int Divide(int numerator, int denominator)
{
  if (denominator == 0)
  {
    throw new ArgumentOutOfRangeException("Denominator cannot be 0.");
  }

  return numerator / denominator;
}

[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void BadParameterTest()
{
  Divide(5, 0);
}

However, verifying that a method implements contract tests is one of the weakest unit tests one can write.

Prove Computational Results

A stronger unit test involves verifying that the computation is correct. It is useful to categorize your methods into one of the three forms of computation:

• Data reduction
• Data transformation
• State change

These determine the kinds of unit tests you might want to write for a particular method.

Data Reduction

The Divide method in the previous sample can be considered a form of data reduction. It takes two values and returns one value. To illustrate:

[TestMethod]
public void VerifyDivisionTest()
{
  Assert.IsTrue(Divide(6, 2) == 3, "6/2 should equal 3!");
}

This is illustrative of testing a method that reduces the inputs, usually, to one resulting output. This is the simplest form of useful unit testing.

Data Transformation

Data transformation unit tests tend to operate on sets of values. For example, the following is a test for a method that converts Cartesian coordinates to polar coordinates.

public static double[] ConvertToPolarCoordinates(double x, double y)
{
  double dist = Math.Sqrt(x * x + y * y);
  double angle = Math.Atan2(y, x);

  return new double[] { dist, angle };
}

[TestMethod]
public void ConvertToPolarCoordinatesTest()
{
  double[] pcoord = ConvertToPolarCoordinates(3, 4);
  Assert.IsTrue(pcoord[0] == 5, "Expected distance to equal 5");
  Assert.IsTrue(pcoord[1] == 0.92729521800161219, "Expected angle to be 53.130
      degrees");
}

This test verifies the correctness of the mathematical transformation.

List Transformations

List transformations should be separated into two tests:

• Verify that the core transformation is correct.
• Verify that the list operation is correct.

For example, from the perspective of unit testing, the following sample is poorly written because it incorporates both the data reduction and the data transformation:

public struct Name
{
  public string FirstName { get; set; }
  public string LastName { get; set; }
}

public List<string> ConcatNames(List<Name> names)
{
  List<string> concatenatedNames = new List<string>();

  foreach (Name name in names)
  {
    concatenatedNames.Add(name.LastName + ", " + name.FirstName);
  }

  return concatenatedNames;
}

[TestMethod]
public void NameConcatenationTest()
{
  List<Name> names = new List<Name>() 
  {
    new Name() { FirstName="John", LastName="Travolta"}, 
    new Name() {FirstName="Allen", LastName="Nancy"}
  };

  List<string> newNames = ConcatNames(names);

  Assert.IsTrue(newNames[0] == "Travolta, John");
  Assert.IsTrue(newNames[1] == "Nancy, Allen");
}

This code is better unit tested by separating the data reduction from the data transformation:

public string Concat(Name name)
{
  return name.LastName + ", " + name.FirstName;
}

[TestMethod]
public void ContactNameTest()
{
  Name name = new Name() { FirstName="John", LastName="Travolta"};
  string concatenatedName = Concat(name);
  Assert.IsTrue(concatenatedName == "Travolta, John");
}

Lambda Expressions and Unit Tests

The Language-Integrated Query (LINQ) syntax is closely coupled with lambda expressions, which results in an easy-to-read syntax that makes life difficult for unit testing. For example, this code:

public List<string> ConcatNamesWithLinq(List<Name> names)
{
  return names.Select(t => t.LastName + ", " + t.FirstName).ToList();
}

is significantly more elegant than the previous examples, but it does not lend itself well to unit testing the actual “unit,” that is, the data reduction from a name structure to a single comma-delimited string expressed in the lambda function t => t.LastName + ", " + t.FirstName. To separate the unit from the list operation requires:

public List<string> ConcatNamesWithLinq(List<Name> names)
{
  return names.Select(t => Concat(t)).ToList();
}

We can see that unit testing can often require refactoring of the code to separate the units from other transformations.

State Change

Most languages are “stateful,” and classes often manage state. The state of a class, represented by its properties, is often a useful thing to test. Consider this class representing the concept of a connection:

public class AlreadyConnectedToServiceException : ApplicationException
{
  public AlreadyConnectedToServiceException(string msg) : base(msg) { }
}

public class ServiceConnection
{
  public bool Connected { get; protected set; }

  public void Connect()
  {
    if (Connected)
    {
      throw new AlreadyConnectedToServiceException("Only one connection at a time is permitted.");
    }

    // Connect to the service.
    Connected = true;
  }

  public void Disconnect()
  {
    // Disconnect from the service.
    Connected = false;
  }
}

We can write unit tests to verify the various permitted and unpermitted states of the object:

[TestClass]
public class ServiceConnectionFixture
{
  [TestMethod]
  public void TestInitialState()
  {
    ServiceConnection conn = new ServiceConnection();
    Assert.IsFalse(conn.Connected);
  }

  [TestMethod]
  public void TestConnectedState()
  {
    ServiceConnection conn = new ServiceConnection();
    conn.Connect();
    Assert.IsTrue(conn.Connected);
  }

  [TestMethod]
  public void TestDisconnectedState()
  {
    ServiceConnection conn = new ServiceConnection();
    conn.Connect();
    conn.Disconnect();
    Assert.IsFalse(conn.Connected);
  }

  [TestMethod]
  [ExpectedException(typeof(AlreadyConnectedToServiceException))]
  public void TestAlreadyConnectedException()
  {
    ServiceConnection conn = new ServiceConnection();
    conn.Connect();
    conn.Connect();
  }
}

Here, each test verifies the correctness of the state of the object:

• When it is initialized.
• When instructed to connect to the service.
• When instructed to disconnect from the service.
• When more than one simultaneous connection is attempted.

State verification often reveals bugs in state management. Also see the following “Mocking Classes” for further improvements to the preceding example code.

Prove a Method Correctly Handles an External Exception

External error handling and recovery is often more important than testing whether your own code generates exceptions at the correct times. There are several reasons for this:

• You have no control over a physically separate dependency, whether it’s a web service, database, or other separate server.
• You have no proof of the correctness of someone else’s code, typically a third-party library.
• Third-party services and software may throw an exception because of a problem that your code is creating but not detecting (and would not necessarily be easy to detect). An example of this is, when deleting records in a database, the database throws an exception because of records in other tables referencing the records your program is deleting, thereby violating a foreign key constraint.

These kinds of exceptions are difficult to test because they require creating at least some error that would be typically generated by the service that you do not control. One way to do this is to “mock” the service; however, this is only possible if the external object is implemented with an interface, an abstract class, or virtual methods.

Mocking Classes

For example, the earlier code for the “ServiceConnection” class is not mockable. If you want to test its state management, you must physically create a connection to the service (whatever that is) that may or may not be available when running the unit tests. A better implementation might look like this:

public class MockableServiceConnection
{
  public bool Connected { get; protected set; }

  protected virtual void ConnectToService()
  {
    // Connect to the service.
  }

  protected virtual void DisconnectFromService()
  {
    // Disconnect from the service.
  }

  public void Connect()
  {
    if (Connected)
    {
      throw new AlreadyConnectedToServiceException("Only one connection at a time is     
      permitted.");
    }

    ConnectToService();
    Connected = true;
  }

  public void Disconnect()
  {
    DisconnectFromService();
    Connected = false;
  }
}

Notice how this minor refactoring now allows you to write a mock class:

public class ServiceConnectionMock : MockableServiceConnection
{
  protected override void ConnectToService()
  {
    // Do nothing.
  }

  protected override void DisconnectFromService()
  {
    // Do nothing.
  }
}

which allows you to write a unit test that tests the state management regardless of the availability of the service. As this illustrates, even simple architectural or implementation changes can greatly improve the testability of a class.

Prove a Bug is Re-creatable

Your first line of defense in proving that the problem has been corrected is, ironically, proving that the problem exists. Earlier we saw an example of writing a test that proved that the Divide method checks for a denominator value of 0. Let’s say a bug report is filed because a user crashed the program when entering 0 for the denominator value.

Negative Testing

The first order of business is to create a test that exercises this condition:

[TestMethod]
[ExpectedException(typeof(DivideByZeroException))]
public void BadParameterTest()
{
  Divide(5, 0);
}

This test passes because we are proving that the bug exists by verifying that when the denominator is 0, a DivideByZeroException is raised. These kinds of tests are considered “negative tests,” as they pass when an error occurs. Negative testing is as important as positive testing (discussed next) because it verifies the existence of a problem before it is corrected.

Prove a Bug is Fixed

Obviously, we want to prove that a bug has been fixed. This is a “positive” test.

Positive Testing

We can now introduce a new test, one that will test that the code itself detects the error by throwing an ArgumentOutOfRangeException.

[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void BadParameterTest()
{
  Divide(5, 0);
}

If we can write this test before fixing the problem, we will see that the test fails. Finally, after fixing the problem, our positive test passes, and the negative test now fails.

While this is a trivial example, it demonstrates two concepts:

• Negative tests—proving that something is repeatedly not working—are important in understanding the problem and the solution.
• Positive tests—proving that the problem has been fixed—are important not only to verify the solution, but also for repeating the test whenever a change is made. Unit testing plays an important role when it comes to regression testing.

Lastly, proving that a bug exists is not always easy. However, as a general rule of thumb, unit tests that require too much setup and mocking are an indicator that the code being tested is not isolated enough from external dependencies and might be a candidate for refactoring.

Prove Nothing Broke When Changing Code

It should be obvious that regression testing is a measurably useful outcome of unit testing. As code undergoes changes, bugs will be introduced that will be revealed if you have good code coverage in your unit tests. This effectively saves considerable time in debugging and more importantly, saves time and money when the programmer discovers the bug rather than the user.

Prove Requirements Are Met

Application development typically starts with a high-level set of requirements, usually oriented around the user interface, workflow, and computations. Ideally, the team reduces the visible set of requirements down to a set of programmatic requirements, which are invisible to the user, by their very nature.

The difference manifests in how the program is tested. Integration testing is typically at the visible level, while unit testing is at the finer grain of invisible, programmatic correctness testing. It is important to keep in mind that unit tests are not intended to replace integration testing; however, just as with high-level application requirements, there are low-level programmatic requirements that can be defined. Because of these programmatic requirements, it is important to write unit tests.

Let’s take a Round method. The .NET Math.Round method will round up a number whose fractional component is greater than 0.5, but will round down when the fractional component is 0.5 or less. Let’s say that is not the behavior we desire (for whatever reason), and we want to round up when the fractional component is 0.5 or greater. This is a computational requirement that should be able to be derived from a higher-level integration requirement, resulting in the following method and test:

public static int RoundUpHalf(double n)
{
  if (n < 0) throw new ArgumentOutOfRangeException("Value must be >= 0.");

  int ret = (int)n;
  double fraction = n - ret;

  if (fraction >= 0.5)
  {
    ++ret;
  }

  return ret;
}

[TestMethod]
public void RoundUpTest()
{
  int result1 = RoundUpHalf(1.5);
  int result2 = RoundUpHalf(1.499999);
  Assert.IsTrue(result1 == 2, "Expected 2.");
  Assert.IsTrue(result2 == 1, "Expected 1.");
}

A separate test for the exception should also be written.

Taking application-level requirements that are verified with integration testing and reducing them to lower-level computational requirements is an important part of the overall unit testing strategy as it defines clear computational requirements that the application must meet. If difficulty is encountered with this process, try to convert the application requirements into one of the three computational categories: data reduction, data transformation, and state change.