CptS 481 - Python Software Construction

Unit 14: Unit and Regression Tests

The doctest Module

Starting with the polar module from last time.

The idea of doctest is to insert tests into docstrings, describing what would appear in an interactive Python shell, with ">>>" denoting inputs, like this:

and we use doctest to run the tests:

Notice: There's no output. That's because no errors were detected. If you want to see the output, including the tests that passed, pass a "-v" to the doctest module:

Let's add a few more tests to the module so that every class and method have tests, as well as the module itself, but this time suppose we're not exactly sure what the output is going to be in some cases.

When we run this, we get:

See the problems? doctest gives us an easy way out: ellipses ("..."), which are like wild card matching. To use them, though, we need to control the doctest invocation (of doctest.testmod()). We'll put it in the modular self-test.

and now the test works:

Let's try a more substantial doctest example: the rational module, which implements a Rational class:

To show another doctest feature, let's start with the faro shuffle program:

Let's write a test for the faro shuffle. Note that we can include error testing.

The doctest for faroShuffle() is thorough, but really long. It clutters up the function. Here's how we make it external. First, move the test "dialog" to a separate file, say faro_shuffle_test.txt:

>>> from faro_doctest import faroShuffle >>> # To shuffle two cards... >>> faroShuffle([0, 1]) [0, 1] >>> # Four cards... >>> faroShuffle([0, 1, 2, 3]) [0, 2, 1, 3] >>> # A deck (range) of 10 cards... >>> faroShuffle(range(10)) [0, 5, 1, 6, 2, 7, 3, 8, 4, 9] >>> # Suppose we try to shuffle a list that includes non-integers... >>> faroShuffle(['queen', 'jack', 'ace', 7]) ['queen', 'ace', 'jack', 7] >>> # Three cards, being an odd number, should raise an exception... >>> faroShuffle([0, 1, 2]) Traceback (most recent call last): ... ValueError: attempt to assign sequence of size 2 to extended slice of size 1 >>> # How about something that's not a sequence? >>> faroShuffle(1) Traceback (most recent call last): ... TypeError: object of type 'int' has no len() >>> # An empty list should be returned unchanged. >>> faroShuffle([]) [] >>> # How about an empty argument list? >>> faroShuffle() Traceback (most recent call last): ... TypeError: faroShuffle() missing 1 required positional argument: 'oldDeck'

The source file then gets much more readable:

The unittest Module

When testing on a larger scale with explicit test code, possibly in files distinct from the module source, there's the unittest module. Here's a simple, self-contained example:

Unit test is all about building test cases.

The unittest.TestCase Class

Let's apply this to faro. Here's a unittest for faro. Notice how we create two test cases as child classes of unittest.TestCase, one for each function in faro.

Note that the methods of the class begin with "test_". This allows us to use unittest.TestCase as a mixin to build testing directly into existing classes. (We don't do that here.)

We need to run this from the shell:

The unittest.FunctionTestCase Class

This class can wrap a single test function and turn it into a test case.

The unittest.TestSuite Class

In a large testing system, it's useful to organize individual tests into "suites" that can be organized by functionality, cost of testing, responsible group, result verbosity, etc. The same test can be included in several groups.

Running doctest Tests in the unittest Framework

It's possble to incorporate doctest tests in the unittest framework. See the book for details.