Chat about Python's unit test framework: unittest

Author: HelloGitHub-Prodesire

Preface

When it comes to Python's unit testing framework, one of the first things that comes to mind for friends who have met Python is unittest.
Indeed, as a standard library for Python, it is excellent and is widely used in various projects.But you know?In fact, among many Python projects, there is more than one mainstream unit testing framework.

This series of articles will introduce the current popular Python unit test framework, describe their functions and features, and compare their similarities and differences, so that you can balance the pros and cons and choose the best unit test framework when facing different scenarios and needs.

This article will take Python 3 as an example by default, and if some features are not or are different in Python 2, they will be specifically described.

1. Introduction

unittest The unit test framework was first inspired by JUnit and has a similar style to the mainstream unit test framework in other languages.

It supports test automation, multiple test cases share setUp and tearDown code, aggregate multiple test cases into a test set, and separate the test and reporting framework.

2. Use Case Writing

The following simple example comes from Official Documents To test three string methods: upper, isupper, split:

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()

In the example above, by inheriting unittest.TestCase To create a test case.
In this class, a method starting with test is defined, and the test framework will execute it as a stand-alone test.

Each use case uses the assertion method built into unittest to determine whether the object under test behaves as expected, such as:

  • In the test_upper test, use assertEqual Check if this is the expected value
  • In the test_isupper test, use assertTrue or assertFalse Verify compliance
  • In the test_split test, use assertRaises Verify that a specific exception was thrown

One might be curious why not use the built-in assert phrase instead of providing so many additional assertion methods and using them?The reason is that by using the assertion method provided by unittest, the test framework is able to aggregate all test results and produce informative test reports when it is finished.While using assert directly can also achieve the purpose of verifying whether the object under test meets expectations, when a use case fails, the error information is not rich enough.

3. Use Case Discovery and Execution

unittest supports use case automatic (recursive) discovery:

  • Default discovery of all test*.py-compliant test cases in the current directory

    • Use python-m unittest or python-m unittest discover
  • The -s parameter specifies the directory to be discovered automatically, and the -p parameter specifies the name pattern of the use case file

    • python -m unittest discover -s project_directory -p "test_*.py"
  • Specify name patterns for automatically discovered directories and use case files through location parameters

    • python -m unittest discover project_directory "test_*.py"

unittest supports execution of specified use cases:

  • Specify test modules

    • python -m unittest test_module1 test_module2
  • Specify Test Class

    • python -m unittest test_module.TestClass
  • Specify test methods

    • python -m unittest test_module.TestClass.test_method
  • Specify test file path (Python 3 only)

    • python -m unittest tests/test_something.py

4. Fixtures

The test fixture is also known as the setUp and tearDown methods.

Test Pre-method setUp() Use to do some preparatory work, such as establishing a database connection.It is called automatically by the framework under test before the case is executed.

Test Cleanup Method tearDown() Use to do some cleanup work, such as disconnecting a database connection.It will be called automatically by the framework under test after the execution of the use case completes, including failures.

Test preemption and cleanup methods can have different levels of execution.

4.1 Effectiveness Level: Test Method

If we want each test method to execute pre-test and clean-up methods before and after, we need to define them in the test class setUp() and tearDown():

class MyTestCase(unittest.TestCase):
    def setUp(self):
        pass

    def tearDown(self):
        pass

4.2 Effective Level: Test Class

If we want a single test class to only execute the precondition once, execute all the tests in that test class, and finally execute the cleanup method, then we need to define it in the test class setUpClass() and tearDownClass():

class MyTestCase(unittest.TestCase):
    def setUpClass(self):
        pass

    def tearDownClass(self):
        pass

4.3 Effective Level: Test Module

If we want a single test module to execute only one precondition, then all tests for all test classes in that module, and finally a cleanup, then we need to define it in the test module setUpModule() and tearDownModule():

def setUpModule():
    pass

def tearDownModule():
    pass

5. Skipping tests and anticipating failures

unittest supports skipping tests directly or conditionally, as well as predicting test failures:

class MyTestCase(unittest.TestCase):

    @unittest.skip("Skip directly")
    def test_nothing(self):
        self.fail("shouldn't happen")

    @unittest.skipIf(mylib.__version__ < (1, 3),
                     "Satisfy Skip")
    def test_format(self):
        # Tests that work for only a certain version of the library.
        pass

    @unittest.skipUnless(sys.platform.startswith("win"), "Satisfy the condition without skipping")
    def test_windows_support(self):
        # windows specific testing code
        pass

    def test_maybe_skipped(self):
        if not external_resource_available():
            self.skipTest("skip")
        # test code that depends on the external resource
        pass

    @unittest.expectedFailure
    def test_fail(self):
        self.assertEqual(1, 0, "This is currently a failure")

6. Subtests

Sometimes you might want to write a test that passes different parameters into a test method to test the same logic, but it will be considered a test, but if you use Subtest Can be considered N (that is, the number of parameters) tests.Here is an example:

class NumbersTest(unittest.TestCase):

    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 6):
            with self.subTest(i=i):
                self.assertEqual(i % 2, 0)

The example uses the method with self.subTest(i=i) to define subtests, in which case, even if a single subtest fails to execute, subsequent subtests will not be affected.In this way, we can see that three of the sub-tests in the output failed:

======================================================================
FAIL: test_even (__main__.NumbersTest) (i=1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "subtests.py", line 32, in test_even
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

======================================================================
FAIL: test_even (__main__.NumbersTest) (i=3)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "subtests.py", line 32, in test_even
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

======================================================================
FAIL: test_even (__main__.NumbersTest) (i=5)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "subtests.py", line 32, in test_even
    self.assertEqual(i % 2, 0)
AssertionError: 1 != 0

7. Output of Test Results

Based on the examples mentioned in the Simple Samples section, explain the result output of the next unittest after running the test.

By default, the output is very simple, showing how many use cases have been run and how much time has been spent:

...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

Detailed output can be obtained by specifying the -v parameter, which displays the use case name in addition to the contents of the default output:

test_isupper (__main__.TestStringMethods) ... ok
test_split (__main__.TestStringMethods) ... ok
test_upper (__main__.TestStringMethods) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

Assuming the test_upper test fails, in verbose output mode, the results are as follows:

test_isupper (tests.test.TestStringMethods) ... ok
test_split (tests.test.TestStringMethods) ... ok
test_upper (tests.test.TestStringMethods) ... FAIL

======================================================================
FAIL: test_upper (tests.test.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Uvsers/prodesire/projects/tests/test.py", line 6, in test_upper
    self.assertEqual('foo'.upper(), 'FOO1')
AssertionError: 'FOO' != 'FOO1'
- FOO
+ FOO1
?    +


----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

If we change self.assertEqual in the test_upper test method to assert, there will be less context information in the test result output that is useful for troubleshooting errors:

test_isupper (tests.test.TestStringMethods) ... ok
test_split (tests.test.TestStringMethods) ... ok
test_upper (tests.test.TestStringMethods) ... FAIL

======================================================================
FAIL: test_upper (tests.test.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/prodesire/projects/tests/test.py", line 6, in test_upper
    assert 'foo'.upper() == 'FOO1'
AssertionError

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)

If you want to generate reports in HTML format, you need additional help with third-party libraries (such as HtmlTestRunner ) to operate.

After installing the third-party libraries, you can't generate HTML reports directly using python-m unittest plus something like -- html report.html. Instead, you need to write a small amount of code yourself to run test cases and get HTML reports.
Check for details Usage instructions for HtmlTestRunner.

8. Summary

unittest As a unit test framework provided by the Python standard library, it is simple to use and powerful, and its daily test requirements can be well met.Without the introduction of third-party libraries, this is the best choice for unit testing.

In the next article, we will introduce the third-party unit test frameworks nose and nose2 and explain how they have improved over unittest so that many developers prefer them.

Explain the Open Source Project Series - Let people who are interested in open source projects stop fearing and the sponsors of open source projects stop being alone.Follow our articles and you'll find it so easy to program, use, and participate in open source projects.Welcome to leave a message to contact us, join us, and let more people love and contribute to open source~

Tags: Python Database Junit Windows

Posted on Mon, 09 Sep 2019 18:13:56 -0700 by Sianide