Test-driven development (TDD) is a process that has been documented considerably over recent years. A process of baking your tests right into your everyday coding, as opposed to a nagging afterthought, should be something that developers seek to make the norm, rather than some ideal fantasy.

I will introduce the core concepts of TDD.

The whole process is very simple to get to grips with, and it shouldn't take too long before you wonder how you were able to get anything done before! There are huge gains to be made from TDD—namely, the quality of your code improving, but also clarity and focus on what it is that you are trying to achieve, and the way in which you will achieve it. TDD also works seamlessly with agile development, and can best be utilized when pair-programming, as you will see later on.

In this tutorial, I will introduce the core concepts of TDD, and will provide examples in Python, using the nosetests unit-testing package. I will additionally offer some alternative packages that are also available within Python.

What Is Test-Driven Development?

This approach allows you to escape the trap that many developers fall into.

TDD, in its most basic terms, is the process of implementing code by writing your tests first, seeing them fail, then writing the code to make the tests pass. You can then build upon this developed code by appropriately altering your test to expect the outcome of additional functionality, then writing the code to make it pass again.

You can see that TDD is very much a cycle, with your code going through as many iterations of tests, writing, and development as necessary, until the feature is finished. By implementing these tests before you write the code, it brings out a natural tendency to think about your problem first. While you start to construct your test, you have to think about the way you design your code. What will this method return? What if we get an exception here? And so on. 

By developing in this way, it means you consider the different routes through the code, and cover these with tests as needed. This approach allows you to escape the trap that many developers fall into (myself included): diving into a problem and writing code exclusively for the first solution you need to handle.

The process can be defined as such:

• Write a failing unit test
• Make the unit test pass
• Refactor

Repeat this process for every feature, as is necessary.

Agile Development With Test-Driven Development

TDD is a perfect match for the ideals and principles of the Agile Development process, with a great striving to deliver incremental updates to a product with true quality, as opposed to quantity. The confidence in your individual units of code that unit testing provides means that you meet this requirement to deliver quality, while eradicating issues in your production environments.

"This means both parties in the pair are engaged, focused on what they are doing, and checking one another's work at every stage."

TDD comes into its own when pair programming, however. The ability to mix up your development workflow, when working as a pair as you see fit, is nice. For example, one person can write the unit test, see it pass, and then allow the other developer to write the code to make the test pass. 

The roles can either be switched each time, each half day, or every day as you see fit. This means both parties in the pair are engaged, focused on what they are doing, and checking each other's work at every stage. This translates to a win in every sense with this approach, I think you'd agree.

TDD also forms an integral part of the Behaviour Driven Development process, which is again, writing tests up front, but in the form of acceptance tests. These ensure a feature "behaves" in the way you expect from end to end. More information can be found an upcoming article here on Tuts+ that will be covering BDD in Python.

Syntax for Unit Testing

The main methods that we make use of in unit testing for Python are:

• assert: base assert allowing you to write your own assertions
• assertEqual(a, b): check a and b are equal
• assertNotEqual(a, b): check a and b are not equal
• assertIn(a, b): check that a is in the item b
• assertNotIn(a, b): check that a is not in the item b
• assertFalse(a): check that the value of a is False
• assertTrue(a): check the value of a is True
• assertIsInstance(a, TYPE): check that a is of type "TYPE"
• assertRaises(ERROR, a, args): check that when a is called with args that it raises ERROR

There are certainly more methods available to us, which you can view—see the Python Unit Test Docs—but, in my experience, the ones listed above are among the most frequently used. We will make use of these within our examples below.

Installing and Using Python's Nose

Before starting the exercises below, you will need to install the nosetest test runner package. Installation of the nosetest runner is straightforward, following the standard "pip" install pattern. It's also usually a good idea to work on your projects using virtualenv's, which keeps all the packages you use for various projects separate. If you are unfamiliar with pip or virtualenv's, you can find documentation on them here: VirtualEnv, PIP.

The pip install is as easy as running this line:

"pip install nose"

Once installed, you can execute a single test file.

$ nosetests example_unit_test.py

Or execute a suite of tests in a folder.

$ nosetests /path/to/tests

The only standard you need to follow is to begin each test's method with “test_” to ensure that the nosetest runner can find your tests!

Options

Some useful command line options that you may wish to keep in mind include:

• -v: gives more verbose output, including the names of the tests being executed.
• -s or -nocapture: allows output of print statements, which are normally captured and hidden while executing tests. Useful for debugging.
• --nologcapture: allows output of logging information.
• --rednose: an optional plugin, which can be downloaded here, but provides colored output for the tests.
• --tags=TAGS: allows you to place an @TAG above a specific test to only execute those, rather than the entire test suite.

Example Problem and Test-Driven Approach

We are going to take a look at a really simple example to introduce both unit testing in Python and the concept of TDD. We will write a very simple calculator class, with add, subtract and other simple methods as you would expect. 

Following a TDD approach, let's say that we have a requirement for an add function, which will determine the sum of two numbers, and return the output. Let's write a failing test for this. 

In an empty project, create two python packages app and test. To make them Python packages (and thus support importing of the files in the tests later on), create an empty file called __init__.py, in each directory. This is Python's standard structure for projects and must be done to allow item to be importable across the directory structure. For a better understanding of this structure, you can refer to the Python packages documentation. Create a file named test_calculator.py in the test directory with the following contents.

import unittest

class TddInPythonExample(unittest.TestCase):

	def test_calculator_add_method_returns_correct_result(self):
		calc = Calculator()
		result = calc.add(2,2)
		self.assertEqual(4, result)

Writing the test is fairly simple.

• First, we import the standard unittest module from the Python standard library.
• Next, we need a class to contain the different test cases.
• Finally, a method is required for the test itself, with the only requirement being that it is named with "test_" at the beginning, so that it may be picked up and executed by the nosetest runner, which we will cover shortly.

With the structure in place, we can then write the test code. We initialize our calculator so that we can execute the methods on it. Following this, we can then call the add method which we wish to test, and store its value in the variable, result. Once this is complete, we can then make use of unittest's assertEqual method to ensure that our calculator's add method behaves as expected.

Now you will use the nosetest runner to execute the test. You could execute the test using the standard unittest runner, if you wish, by adding the following block of code to the end of your test file.

if __name__ == '__main__':
    unittest.main()

This will allow you to run the test using the standard way of executing Python files, $ python test_calculator.py. However, for this tutorial you are going to make use of the nosetests runner, which has some nice features such as being able to execute nose tests against a directory and running all tests, amongst other useful features.

$ nosetests test_calculator.py
E
======================================================================
ERROR: test_calculator_add_method_returns_correct_result (test.test_calculator.TddInPythonExample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/user/PycharmProjects/tdd_in_python/test/test_calculator.py", line 6, in test_calculator_add_method_returns_correct_result
    calc = Calculator()
NameError: global name 'Calculator' is not defined

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

From the output nosetest has given us, we can see that the problem relates to us not importing Calculator. That's because we haven't created it yet! So let's go and define our Calculator in a file named calculator.py under the app directory and import it:

	class Calculator(object):

		def add(self, x, y):
			pass

import unittest
from app.calculator import Calculator

class TddInPythonExample(unittest.TestCase):

	def test_calculator_add_method_returns_correct_result(self):
		calc = Calculator()
		result = calc.add(2,2)
		self.assertEqual(4, result)


if __name__ == '__main__':
    unittest.main()

Now that we have Calculator defined, let's see what nosetest indicates to us now:

$ nosetests test_calculator.py
F
======================================================================
FAIL: test_calculator_add_method_returns_correct_result (test.test_calculator.TddInPythonExample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/user/PycharmProjects/tdd_in_python/test/test_calculator.py", line 9, in test_calculator_add_method_returns_correct_result
    self.assertEqual(4, result)
AssertionError: 4 != None

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1)

So, obviously, our add method is returning the wrong value, as it doesn't do anything at the moment. Handily, nosetest gives us the offending line in the test, and we can then confirm what we need to change. Let's fix the method and see if our test passes now:

	class Calculator(object):

		def add(self, x, y):
			return x+y

$ nosetests test_calculator.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Success! We have defined our add method and it works as expected. However, there is more work to do around this method to ensure that we have tested it properly.

We have fallen into the trap of just testing the case we are interested in at the moment.

What would happen if someone were to add anything other than numbers? Python will actually allow for the addition of strings and other types, but in our case, for our calculator, it makes sense to only allow adding of numbers. Let's add another failing test for this case, making use of the assertRaises method to test if an exception is raised here:

import unittest
from app.calculator import Calculator


class TddInPythonExample(unittest.TestCase):

    def setUp(self):
        self.calc = Calculator()

    def test_calculator_add_method_returns_correct_result(self):
        result = self.calc.add(2, 2)
        self.assertEqual(4, result)

    def test_calculator_returns_error_message_if_both_args_not_numbers(self):
        self.assertRaises(ValueError, self.calc.add, 'two', 'three')


if __name__ == '__main__':
    unittest.main()

You can see from above that we added the test and are now checking for a ValueError to be raised, if we pass in strings. We could also add more checks for other types, but for now, we'll keep things simple. You may also notice that we've made use of the setup() method. This allows us to put things in place before each test case. So, as we need our Calculator object to be available in both test cases, it makes sense to initialize this in the setUp method. Let's see what nosetest indicates to us now:

$ nosetests test_calculator.py
.F
======================================================================
FAIL: test_calculator_returns_error_message_if_both_args_not_numbers (test.test_calculator.TddInPythonExample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/user/PycharmProjects/tdd_in_python/test/test_calculator.py", line 15, in test_calculator_returns_error_message_if_both_args_not_numbers
    self.assertRaises(ValueError, self.calc.add, 'two', 'three')
AssertionError: ValueError not raised

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

Clearly, nosetests indicates to us that we are not raising the ValueError when we expect to be. Now that we have a new failing test, we can code the solution to make it pass.

class Calculator(object):
    def add(self, x, y):
        number_types = (int, long, float, complex)

        if isinstance(x, number_types) and isinstance(y, number_types):
            return x + y
        else:
            raise ValueError

From the code above, you can see that we've added a small addition to check the types of the values and whether they match what we want. One approach to this problem could mean that you follow duck typing, and simply attempt to use it as a number, and "try/except" the errors that would be raised in other cases. The above is a bit of an edge case and means we must check before moving forward. As mentioned earlier, strings can be concatenated with the plus symbol, so we only want to allow numbers. Utilising the isinstance method allows us to ensure that the provided values can only be numbers.

To complete the testing, there are a couple of different cases that we can add. As there are two variables, it means that both could potentially not be numbers. Add the test case to cover all the scenarios.

import unittest
from app.calculator import Calculator


class TddInPythonExample(unittest.TestCase):
    def setUp(self):
        self.calc = Calculator()

    def test_calculator_add_method_returns_correct_result(self):
        result = self.calc.add(2, 2)
        self.assertEqual(4, result)

    def test_calculator_returns_error_message_if_both_args_not_numbers(self):
        self.assertRaises(ValueError, self.calc.add, 'two', 'three')

    def test_calculator_returns_error_message_if_x_arg_not_number(self):
        self.assertRaises(ValueError, self.calc.add, 'two', 3)

    def test_calculator_returns_error_message_if_y_arg_not_number(self):
        self.assertRaises(ValueError, self.calc.add, 2, 'three')


if __name__ == '__main__':
    unittest.main()

When we run all these tests now, we can confirm that the method meets our requirements!

$ nosetests test_calculator.py
....
----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

Other Unit Test Packages

py.test

This is a similar test runner to nosetest, which makes use of the same conventions, meaning that you can execute your tests in either of the two. A nice feature of pytest is that it captures your output from the test at the bottom in a separate area, meaning you can quickly see anything printed to the command line (see below). I've found pytest to be useful when executing single tests, as opposed to a suite of tests.

To install the pytest runner, follow the same pip install procedure that you followed to install nosetest. Simply execute $ pip install pytest and it will grab the latest version and install to your machine. You can then execute the runner against your suite of tests by providing the directory of your test files, $ py.test test/, or you can provide the path to the test file you wish to execute: $ py.test test/calculator_tests.py.

$ py.test test/test_calculator.py
================================================================= test session starts =================================================================
platform darwin -- Python 2.7.6 -- py-1.4.26 -- pytest-2.6.4
collected 4 items 

test/test_calculator.py ....

============================================================== 4 passed in 0.02 seconds ===============================================================

An example of pytest's output when printing from within your tests or code is shown below. This can be useful for quickly debugging your tests and seeing some of the data it is manipulating. NOTE: you will only be shown output from your code on errors or failures in your tests, otherwise pytest suppresses any output.

$ py.test test/test_calculator.py 
================================================================= test session starts =================================================================
platform darwin -- Python 2.7.6 -- py-1.4.26 -- pytest-2.6.4
collected 4 items 

test/test_calculator.py F...

====================================================================== FAILURES =======================================================================
________________________________________ TddInPythonExample.test_calculator_add_method_returns_correct_result _________________________________________

self = <test.test_calculator.TddInPythonExample testMethod=test_calculator_add_method_returns_correct_result>

    def test_calculator_add_method_returns_correct_result(self):
        result = self.calc.add(3, 2)
>       self.assertEqual(4, result)
E       AssertionError: 4 != 5

test/test_calculator.py:11: AssertionError
---------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------
X value is: 3
Y value is: 2
Result is 5
========================================================= 1 failed, 3 passed in 0.03 seconds ==========================================================

UnitTest

Python's inbuilt unittest package that we have used to create our tests can actually be executed, itself, and gives nice output. This is useful if you don't wish to install any external packages and keep everything pure to the standard library. To use this, simply add the following block to the end of your test file.

if __name__ == '__main__':
    unittest.main()

Execute the test using python calculator_tests.py. Here is the output that you can expect:

$ python test/test_calculator.py 
....
----------------------------------------------------------------------
Ran 4 tests in 0.004s

OK

Debug Code With PDB

Often when following TDD, you will encounter issues with your code and your tests will fail. There will be occasions where, when your tests do fail, it isn't immediately obvious why that is happening. In such instances, it will be necessary to apply some debugging techniques to your code to understand exactly how the code is manipulating the data and not getting the exact response or outcome that you expect.

There will be occasions where, when your tests do fail, it isn't immediately obvious why that is happening

Fortunately, when you find yourself in such a position, there are a couple of approaches you can take to understand what the code is doing and rectify the issue to get your tests passing. The simplest method, and one many beginners use when first writing Python code, is to add print statements at certain points in your code and see what they output when running tests. 

Debug With Print Statements

If you deliberately alter our calculator code so that it fails, you can get an idea of how debugging your code will work. Change the code in the add method of app/calculator.py to actually subtract the two values.

class Calculator(object):
    def add(self, x, y):
        number_types = (int, long, float, complex)

        if isinstance(x, number_types) and isinstance(y, number_types):
            return x - y
        else:
            raise ValueError

When you run the tests now, the test which checks that your add method correctly returns four when adding two plus two fails, as it now returns 0. To check how it is reaching this conclusion, you could add some print statements to check that it is receiving the two values correctly and then check the output. This would then lead you to conclude the logic on the addition of the two numbers is incorrect. Add the following print statements to the code in app/calculator.py.

class Calculator(object):
    def add(self, x, y):
        number_types = (int, long, float, complex)

        if isinstance(x, number_types) and isinstance(y, number_types):
            print 'X is: {}'.format(x)
            print 'Y is: {}'.format(y)
            result = x - y
            print 'Result is: {}'.format(result)
            return result
        else:
            raise ValueError

Now when you execute nosetest against the tests, it nicely shows you the captured output for the failing test, giving you a chance to understand the problem and fix the code to make the addition rather than subtraction.

$ nosetests test/test_calculator.py
F...
======================================================================
FAIL: test_calculator_add_method_returns_correct_result (test.test_calculator.TddInPythonExample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/user/PycharmProjects/tdd_in_python/test/test_calculator.py", line 11, in test_calculator_add_method_returns_correct_result
    self.assertEqual(4, result)
AssertionError: 4 != 0
-------------------- >> begin captured stdout << ---------------------
X is: 2
Y is: 2
Result is: 0

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 4 tests in 0.002s

FAILED (failures=1)

Advanced Debug With PDB

As you start to write more advanced code, print statements alone will not be enough or start to become tiresome to write all over the place and have to be cleaned up later. As the process of needing to debug has become commonplace when writing code, tools have evolved to make debugging Python code easier and more interactive.

One of the most commonly used tools is pdb (or Python Debugger). The tool is included in the standard library and simply requires adding one line where you would like to stop the program execution and enter into pdb, typically known as the "breakpoint". Using our failing code in the add method, try adding the following line before the two values are subtracted.

class Calculator(object):
    def add(self, x, y):
        number_types = (int, long, float, complex)

        if isinstance(x, number_types) and isinstance(y, number_types):
            import pdb; pdb.set_trace()
            return x - y
        else:
            raise ValueError

If using nosetest to execute the test, be sure to execute using the -s flag which tells nosetest to not capture standard output, otherwise your test will just hang and not give you the pdb prompt. Using the standard unittest runner and pytest does not require such a step.

With the pdb code snippet in place, when you execute the test now, the execution of the code will break at the point at which you placed the pdb line, and allow you to interact with the code and variables that are currently loaded at the point of execution. When the execution first stops and you are given the pdb prompt, try typing list to see where you are in the code and what line you are currently at.

$ nosetests -s
> /Users/user/PycharmProjects/tdd_in_python/app/calculator.py(7)add()
-> return x - y
(Pdb) list
  2          def add(self, x, y):
  3  	        number_types = (int, long, float, complex)
  4  	
  5  	        if isinstance(x, number_types) and isinstance(y, number_types):
  6  	            import pdb; pdb.set_trace()
  7  ->	            return x - y
  8  	        else:
  9  	            raise ValueError
[EOF]
(Pdb) 

You can interact with your code, as if you were within a Python prompt, so try evaluating what is in the x and y variables at this point.

(Pdb) x
2
(Pdb) y
2

You can continue to "play" around with the code as you require to figure out what is wrong. You can type help at any point to get a list of commands, but the core set you will likely need are:

• n: step forward to next line of execution.
• list: show five lines either side of where you are currently executing to see the code involved with the current execution point.
• args: list the variables involved in the current execution point.
• continue: run the code through to completion.
• jump <line number>: run the code until the specified line number.
• quit/exit: stop pdb.

Conclusion

Test-Driven Development is a process that can be both fun to practice, and hugely beneficial to the quality of your production code. Its flexibility in its application to anything from large projects with many team members right down to a small solo project means that it's a fantastic methodology to advocate to your team.

Whether pair programming or developing by yourself, the process of making a failing test pass is hugely satisfying. If you've ever argued that tests weren't necessary, hopefully this article has swayed your approach for future projects.

Make TDD a part of your daily workflow today.

Heads Up!

If this article has whetted your appetite for the world of testing in Python, why not check out the book "Testing Python" written by the articles author and released on Amazon and other good retailers recently. Visit this page to purchase your copy of the book today, and support one of your Tuts+ contributors.