This is an extract from the Unit Testing Succinctly eBook, by Marc Clifton, kindly provided by Syncfusion.

Unit testing is all about proving correctness. To prove that something is working correctly, you first have to understand what both a unit and a test actually are before you can explore what is provable within the capabilities of unit testing.

What Is a Unit?

In the context of unit testing, a unit has several characteristics.

Pure Units

A pure unit is the easiest and most ideal method for writing a unit test. A pure unit has several characteristics that facilitate easy testing.

A Unit Should (Ideally) Not Call Other Methods

With regard to unit testing, a unit should first and foremost be a method that does something without calling any other methods. Examples of these pure units can be found in the String and Math classes—most of the operations performed do not rely on any other method. For example, the following code (taken from something the author has written)

public void SelectedMasters()
{
  string currentEntity = dgvModel.DataMember;
  string navToEntity = cbMasterTables.SelectedItem.ToString();
  DataGridViewSelectedRowCollection selectedRows = dgvModel.SelectedRows;

  StringBuilder qualifier = BuildQualifier(selectedRows);
  UpdateGrid(navToEntity);
  SetRowFilter(navToEntity, qualifier.ToString());
  ShowNavigateToMaster(navToEntity, qualifier.ToString());
}

should not be considered a unit for three reasons:

• Rather than taking parameters, it obtains the values involved in the computation from user interface objects, specifically a DataGridView and a ComboBox.
• It makes several calls to other methods that potentially are units.
• One of the methods appears to update the display, entangling a computation with a visualization.

The first reason points out a subtle issue—properties should be considered method calls. In fact, they are in the underlying implementation. If your method is using properties of other classes, this is a kind of method call and should be considered carefully when writing a suitable unit.

Realistically, this is not always possible. Often enough, a call to the framework or some other API is required for the unit to successfully do its work. However, these calls should be inspected to determine whether the method could be improved to make a purer unit, for example, by extracting the calls into a higher method and passing the results of the calls as a parameter to the unit.

A Unit Should Do Only One Thing

A corollary to “a unit should not call other methods” is that a unit is a method that does one thing and one thing only. Often other methods are called in order to do more than one thing—a valuable skill to know when something actually consists of several subtasks—even if it can be described as a high-level task, which makes it sound like a single task!

The following code might look like a reasonable unit that does one thing: it inserts a name into the database.

public int Insert(Person person)
{
  DbProviderFactory factory = SqlClientFactory.Instance;

  using (DbConnection connection = factory.CreateConnection())
  {
    connection.ConnectionString = "Server=localhost; Database=myDataBase;
        Trusted_Connection=True;";
    connection.Open();

    using (DbCommand command = connection.CreateCommand())
    {
      command.CommandText = "insert into PERSON (ID, NAME) values (@Id, @Name)";
      command.CommandType = CommandType.Text;

      DbParameter id = command.CreateParameter();
      id.ParameterName = "@Id";
      id.DbType = DbType.Int32;
      id.Value = person.Id;

      DbParameter name = command.CreateParameter();
      name.ParameterName = "@Name";
      name.DbType = DbType.String;
      name.Size = 50;
      name.Value = person.Name;

      command.Parameters.AddRange(new DbParameter[] { id, name });

      int rowsAffected = command.ExecuteNonQuery();

      return rowsAffected;
    }
  }
}

However, this code is actually doing several things:

• Obtaining a SqlClient factory provider instance.
• Instantiating a connection and opening it.
• Instantiating a command and initializing the command.
• Creating and adding two parameters to the command.
• Executing the command and returning the number of rows affected.

There are a variety of issues with this code that disqualify it from being a unit and make it difficult to reduce into basic units. A better way to write this code might look like this:

public int RefactoredInsert(Person person)
{
  DbProviderFactory factory = SqlClientFactory.Instance;
  using (DbConnection conn = OpenConnection(factory, "Server=localhost; Database=myDataBase;
     Trusted_Connection=True;"))
  {
    using (DbCommand cmd = CreateTextCommand(conn, "insert into PERSON (ID, NAME) values (@Id, @Name)"))
    {
      AddParameter(cmd, "@Id", person.Id);
      AddParameter(cmd, "@Name", 50, person.Name);
      int rowsAffected = cmd.ExecuteNonQuery();

      return rowsAffected;
    }
  }
}

protected DbConnection OpenConnection(DbProviderFactory factory, string connectString)
{
  DbConnection conn = factory.CreateConnection();
  conn.ConnectionString = connectString;
  conn.Open();

  return conn;
}

protected DbCommand CreateTextCommand(DbConnection conn, string cmdText)
{
  DbCommand cmd = conn.CreateCommand();
  cmd.CommandText = cmdText;
  cmd.CommandType = CommandType.Text;

  return cmd;
}

protected void AddParameter(DbCommand cmd, string paramName, int paramValue)
{
  DbParameter param = cmd.CreateParameter();
  param.ParameterName = paramName;
  param.DbType = DbType.Int32;
  param.Value = paramValue;
  cmd.Parameters.Add(param);
}

protected void AddParameter(DbCommand cmd, string paramName, int size, string paramValue)
{
  DbParameter param = cmd.CreateParameter();
  param.ParameterName = paramName;
  param.DbType = DbType.String;
  param.Size = size;
  param.Value = paramValue;
  cmd.Parameters.Add(param);
}

Notice how, in addition to looking cleaner, the methods OpenConnection, CreateTextCommand, and AddParameter are more suitable to unit testing (ignoring the fact that they are protected methods). These methods do only one thing and, as units, can be tested to ensure that they do that one thing correctly. From this, there becomes little point to testing the RefactoredInsert method, as it relies entirely on other functions that have unit tests. At best, one might want to write some exception handling test cases, and possibly some validation on the fields in the Person table.

Provably Correct Code

What if the higher-level method does something more than just call other methods for which there are unit tests, say, some sort of additional computation? In that case, the code performing the computation should be moved to its own method, tests should be written for it, and again the higher-level method can rely on the correctness of the code it calls. This is the process of constructing provably correct code. The correctness of higher-level methods improves when all they do is call lower-level methods that have proofs (unit tests) of correctness.

A Unit Should Not (Ideally) Have Multiple Code Paths

Cyclomatic complexity is the bane of unit testing and application testing in general, as it increases the difficulty of testing all the code paths. Ideally, a unit will not have any if or switch statements. The body of those statements should be regarded as the units (assuming they meet the other criteria of a unit) and to be made testable, should be extracted into their own methods.

Here is another example taken from the author’s MyXaml project (part of the parser):

if (tagName=="*")
{
  foreach (XmlNode node in topElement.ChildNodes)
  {
    if (!(node is XmlComment))
    {
      objectNode = node;
      break;
    }
  }

  foreach (XmlAttribute attr in objectNode.Attributes)
  {
    if (attr.LocalName == "Name")
    {
      nameAttr = attr;    
      break;
    }
  }
}
else
{
  ... etc ...
}

Here we have multiple code paths involving if, else, and foreach statements, which:

• Create setup complexity, as many conditions must be met to execute the inner code.
• Create testing complexity, as the code paths require different setups to ensure that each code path is tested.

Obviously, conditional branching, loops, case statements, etc. cannot be avoided, but it may be worthwhile to consider refactoring the code so that the internals of the conditions and loops are separate methods that can be independently tested. Then the tests for the higher-level method can simply ensure that the states (represented by conditions, loops, switches, etc.) are properly handled, independent of the computations that they perform.

Dependent Units

Methods that have dependencies on other classes, data, and state information are more complex to test because those dependencies translate into requirements for instantiated objects, existence of data, and predetermined state.

Preconditions

In its simplest form, dependent units have preconditions that must be met. Unit test engines provide mechanisms to instantiate test dependencies, both for individual tests and for all tests within a test group, or “fixture.”

Actual or Simulated Services

Complicated dependent units require services such as database connections to be instantiated or simulated. In the earlier code example, the Insert method cannot be unit tested without the ability to connect to an actual database. This code becomes more testable if the database interaction can be simulated, typically through the use of interfaces or base classes (abstract or not).

The refactored methods in the Insert code described earlier are a good example because DbProviderFactory is an abstract base class, so one can easily create a class deriving from DbProviderFactory to simulate the database connection.

Handling External Exceptions

Dependent units, because they are making calls to other APIs or methods, are also more fragile—they may need to explicitly handle errors potentially generated by the methods that they call. In the earlier code sample, the Insert method’s code could be wrapped in a try-catch block, because it is certainly possible that the database connection may not exist. The exception handler might return 0 for the number of rows affected, reporting the error through some other mechanism. In such a scenario, the unit tests must be capable of simulating this exception to ensure that all code paths are executed correctly, including catch and finally blocks.

What is a Test?

A test provides a useful assertion of the correctness of the unit. Tests that assert the correctness of a unit typically exercise the unit in two ways:

• Testing how the unit behaves under normal conditions.
• Testing how the unit behaves under abnormal conditions.

Normal Conditions Testing

Testing how the unit behaves under normal conditions is by far the easiest test to write. After all, when we write a function, we are either writing it to satisfy an explicit or implicit requirement. The implementation reflects an understanding of that requirement, which in part encompasses what we expect as inputs to the function and how we expect the function to behave with those inputs. Therefore, we are testing the result of the function given expected inputs, whether the result of the function is a return value or a state change. Furthermore, if the unit is dependent on other functions or services, we are also expecting them to behave correctly and are writing a test with that implied assumption.

Abnormal Conditions Testing

Testing how the unit behaves under abnormal conditions is much more difficult. It requires determining what an abnormal condition is, which is usually not obvious by inspecting the code. This is made more complicated when testing a dependent unit—a unit that is expecting another function or service to behave correctly. In addition, we don’t know how another programmer or user might exercise the unit.

Unit Tests and Other Testing Practices

Unit testing does not replace other testing practices; it should complement other testing practices, providing additional documentation support and confidence. Figure 1 illustrates one concept of the "application development flow"—how other testing integrates with unit testing. Note that the customer can be involved in any stage, though usually at the acceptance test procedure (ATP), system integration, and usability stages.

Compare this with the V-model of the software development and testing process. While it is related to the waterfall model of software development (which, ultimately, all other software development models are either a subset or an extension of), the V-model provides a good picture of what kind of testing is required for each layer of the software development process:

Furthermore, when a test point fails in some other test practice, a specific piece of code can usually be identified as being responsible for the failure. When that is the case, it becomes possible to treat that piece of code as a unit and write a unit test to first create the failure and, when the code has been changed, to verify the fix.

Acceptance Test Procedures

An acceptance test procedure (ATP) is often used as a contractual requirement to prove that certain functionality has been implemented. ATPs are often associated with milestones, and milestones are often associated with payments or further project funding. An ATP differs from a unit test because the ATP demonstrates that the functionality with respect to the whole line-item requirement has been implemented. For example, a unit test can determine whether the computation is correct. However, the ATP might validate that the user elements are provided in the user interface and that the user interface displays the result of the computation as specified by the requirement. These requirements are not covered by the unit test.

Automated User Interface Testing

An ATP might initially be written as a series of user interface (UI) interactions to verify that the requirements have been met. Regression testing of the application as it continues to evolve is applicable to unit testing as well as acceptance testing. Automated user interface testing is another tool completely separate from unit testing that saves time and manpower, while reducing testing errors. As with ATPs, unit tests in no way replace the value of automated user interface tests.

Usability and User Experience Testing

Unit tests, ATPs, and automated UI tests do not in any way replace usability testing—putting the application in front of users and getting their "user experience" feedback. Usability testing should not be about finding computational defects (bugs), and therefore is completely outside of the purview of unit tests.

Performance and Load Testing

Some unit test tools provide a means for measuring the performance of a method. For example, Visual Studio’s test engine reports on execution time, and NUnit has attributes that can be used to verify that a method executes within an allotted time.

Ideally, a unit test tool for .NET languages should explicitly implement performance testing to compensate for just-in-time (JIT) code compilation the first time the code is executed.

Most load tests (and the related performance tests) are not suitable for unit tests. Certain forms of load tests can be done with unit testing as well, at least to the limitations of the hardware and operating system, such as:

• Simulating memory constraints.
• Simulating resource constraints.

However, these kinds of tests ideally require the support of the framework or OS API to simulate these kinds of loads for the application being tested. Forcing the entire OS to consume a large amount of memory, resources, or both, affects all the applications, including the unit test application. This is not a desirable approach.

Other types of load testing, such as simulating multiple instances of running an operation simultaneously, are not candidates for unit testing. For example, testing the performance of a web service with a load of one million transactions per minute is probably not possible using a single machine. While this kind of test can be easily written as a unit, the actual test would involve a suite of test machines. And in the end, you’ve only tested a very narrow behavior of the web service under very specific network conditions, which in no way actually represent the real world.

For this reason, performance and load testing have limited application with unit testing.