Pytest is a popular test framework in Python. It helps to automate the test execution process and run unit tests as frequently as possible with minimal effort.
Everyone who has at least some experience with computer programming would intuitively know that testing is critical when building a software application. But beginners often find it difficult to know where and how to start. If that is you, this article will help you get started.
Why should I use Pytest?
Pytest is probably the best option for testing, not only for beginners but for any Python developers. Python includes a standard test framework called unittest, a very powerful test framework. But Pytest has the following advantages among others:
Easy to use
Pytest can do many things, but you can use basic functionality right out of the box because it comes with so-called sensible defaults. It will automatically find your test files and execute the test cases. You can also write test assertions simply by using the Python assertunittest, for example, where you would need to use different assert methods, such as assertEqual() or assertFalse(). You will see some examples of assertions in Pytest later in this article.
Rich ecosystem of plugins
Pytest has a rich third-party plugin ecosystem. Plugins can enhance the capability of Pytest and help you write and run tests more efficiently. Some plugins focus on specific areas, such as Django or Flask, but others are more generic. At the time of writing, there are more than 300 plugins available, so whatever application you are developing, you will be able to find plugins that suit your needs.
Compatible with unittest
Pytest can run tests written in the unittest style, so, for example, if you already have unittest test files, you can continue using them with Pytest. But if you want to utilise the full functionality of Pytest, you will need to write tests in the Pytest style. At the same time, you can also use the functionality of unittest, such as Mock, in Pytest.
Ultimately, which tool to use will largely depend on personal preferences (if you have a choice). But the current popularity of Pytest shows that I am not a minority, so I would recommend trying it out if you have not used it yet.
How can I install Pytest?
Let’s start by installing Pytest. Like other Python packages, you can install Pytest from PyPI (Python Package Index) simply using the pip command.
Open a terminal (Mac and Linux) or command prompt (Windows) and type the following command:
$ pip install pytest
Once it has finished, you can check the installation by running pytest --version. If you see the version number, you are all set. The version number will depend on when you install the package.
$ pytest --version pytest 6.2.5
How can I write a test in Pytest?
The simplest way to write a test in Pytest is to write it in the same source file.
Assume that you have a Python file called calc.py. In this file, you have a function called mul(), which takes two integers as arguments and returns the multiplication of the two values.
def mul(a: int, b: int) -> int:
return a * bYou can check the output of this function by manually running the function on the Python interpreter.
$ python Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec 7 2020, 12:44:01) [Clang 12.0.0 (clang-1200.0.32.27)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import calc >>> calc.mul(2, 3) 6
It works. But, as your application grows, it quickly becomes challenging to check everything in this way manually. So, let’s use Pytest to make this process easier.
Write a function test_mul_should_succeed() in the same calc.py file.
def mul(a: int, b: int) -> int:
return a * b
def test_mul_should_succeed_with_int_params() -> None:
result = mul(2, 3)
expected = 6
assert result == expectedThe function name should start with the prefix test_. It is because Pytest finds test functions beginning with this prefix by default. Also it is helpful to name the function so that you can see what kind of tests the function executes when you look at the function name. Pytest can show the function names and their test results in the output, so it becomes easier to know which tests have failed, as you will see later in this article.
In the test function body, you get the result from the target function (the function you test) and then compare it with the expected value using the assert statement. It returns True when the expression specified after the assert keyword is True. In this case, we expect mul(2, 3) to return 6, so this test should pass. Let’s check it out.
How can I run Pytest?
You can run tests using the pytest command.
Go back to the terminal and run the command as shown below.
$ pytest calc.py =============== test session starts =============== platform darwin -- Python 3.9.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 rootdir: /Users/mikio/pytest collected 1 item calc.py . [100%] ================ 1 passed in 0.00s ================
You can see that the file name (calc.py) and a dot (.) in the output. It means that Pytest ran one test in the file calc.py, and the test passed. As we only have one test, 100% of the tests have passed.
calc.py . [100%]
As this output is not very informative, let’s add the -v option and run the pytest command again.
$ pytest -v calc.py =============== test session starts =============== platform darwin -- Python 3.9.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /Users/mikio/pytest/venv/bin/python3 cachedir: .pytest_cache rootdir: /Users/mikio/pytest collected 1 item calc.py::test_mul_should_succeed_with_int_params PASSED [100%] ================ 1 passed in 0.00s ================
This time, the output has more information, and you can see the function name and the result (PASSED).
calc.py::test_mul_should_succeed_with_int_params PASSED [100%]
You can add as many assert statements as you like to the function. (The assert has been rewritten for conciseness.)
def mul(a: int, b: int) -> int:
return a * b
def test_mul_should_succeed_with_int_params() -> None:
assert mul(2, 3) == 6
assert mul(5, 4) == 20
assert mul(-1, 1) == -1
As long as all asserts pass, the function passes.
$ pytest -v calc.py =============== test session starts =============== platform darwin -- Python 3.9.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /Users/mikio/pytest/venv/bin/python3 cachedir: .pytest_cache rootdir: /Users/mikio/pytest collected 1 item calc.py::test_mul_should_succeed_with_int_params PASSED [100%] ================ 1 passed in 0.00s ================
If one of the assert statements fails, the function fails. Now let’s assume that the requirements for the function mul() have slightly changed, and now the function might get string arguments. Add another test function to check that the function returns a correct result when one of the arguments is a string.
from typing import Union
def mul(a: Union[int, str], b: Union[int, str]) -> int:
return a * b
def test_mul_should_succeed_with_int_params() -> None:
assert mul(2, 3) == 6
assert mul(5, 4) == 20
assert mul(-1, 1) == -1
def test_mul_should_succeed_with_str_params() -> None:
assert mul('1', 1) == 1
This test fails, but Pytest shows you exactly which assert statement failed with the actual and expected values, which is very helpful for the problem analysis.
$ pytest -v calc.py
=============== test session starts ===============
platform darwin -- Python 3.9.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /Users/mikio/pytest/venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/mikio/pytest
collected 2 items
calc.py::test_mul_should_succeed_with_int_params PASSED [ 50%]
calc.py::test_mul_should_succeed_with_str_params FAILED [100%]
==================== FAILURES =====================
_____ test_mul_should_succeed_with_str_params _____
def test_mul_should_succeed_with_str_params() -> None:
> assert mul('1', 1) == 1
E AssertionError: assert '1' == 1
E +'1'
E -1
calc.py:10: AssertionError
============= short test summary info =============
FAILED calc.py::test_mul_should_succeed_with_str_params
=========== 1 failed, 1 passed in 0.02s ===========
In this case, let’s convert the input arguments to integer.
from typing import Union
def mul(a: Union[int, str], b: Union[int, str]) -> int:
return int(a) * int(b)
def test_mul_should_succeed_with_int_params() -> None:
assert mul(2, 3) == 6
assert mul(5, 4) == 20
assert mul(-1, 1) == -1
def test_mul_should_succeed_with_str_params() -> None:
assert mul('1', 1) == 1
Now the test passes.
$ pytest -v calc.py =============== test session starts =============== platform darwin -- Python 3.9.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /Users/mikio/pytest/venv/bin/python3 cachedir: .pytest_cache rootdir: /Users/mikio/pytest collected 2 items calc.py::test_mul_should_succeed_with_int_params PASSED [ 50%] calc.py::test_mul_should_succeed_with_str_params PASSED [100%] ================ 2 passed in 0.00s ================
How should I organise tests?
Although writing tests in the same source file works perfectly fine, as shown in the previous section, it will quickly become difficult to manage tests as your application becomes more complex.
It is common practice in real-world use cases to create a separate test file for each source file. If you have many source files, you might want to create a directory and put all the test files there. But as we only have one source file, let’s create a file called test_calc.py in the same directory and move the test function into this file.
Like in the function name, it is important to have the prefix test_ in the test file name because Pytest automatically discovers test files with this prefix by default. Note that you need to import the function mul() from calc.py because now the test functions are defined in a separate Python file.
calc.py
from typing import Union
def mul(a: Union[int, str], b: Union[int, str]) -> int:
return int(a) * int(b)
test_calc.py
from calc import mul
def test_mul_should_succeed_with_int_params() -> None:
assert mul(2, 3) == 6
assert mul(5, 4) == 20
assert mul(-1, 1) == -1
def test_mul_should_succeed_with_str_params() -> None:
assert mul('1', 1) == 1
Now go back to the terminal and run pytest. This time, you don’t even need to specify the Python file as the command line argument because Pytest will automatically discover test files in the current directory by default.
$ pytest =============== test session starts =============== platform darwin -- Python 3.9.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 rootdir: /Users/mikio/pytest collected 2 items test_calc.py .. [100%] ================ 2 passed in 0.01s ================
How to parametrize tests in Pytest?
You can add as many assert statements with different input values as you like, but it creates repetition. For example, if you needed to change the function name from mul() to mul_v1() for some reason, you would need to change the function name in all assert statements, which can be error-prone.
You can use the pytest.mark.parametrize decorator to solve this problem. There are three things to change:
- Firstly, you need to import pytest in your test file.
- Secondly, add
@pytest.mark.parametrizedecorator to the test function. The decorator has two arguments:- The first argument is the string representation of the parameter names, separated by a comma (,).
- The second argument is a list of tuples. In each tuple, specify the values of the parameters in the same order as specified in the first argument.
- Lastly, specify the parameter names in the test function arguments. Then these parameters become available within the test function, and you can use them in the assert statements.
The following shows the parametrised version of the test functions in the previous section.
import pytest
from calc import mul
@pytest.mark.parametrize(
"a,b,expected",
[(2, 3, 6), (5, 4, 20), (-1, 1, -1)]
)
def test_mul_should_succeed_with_int_params(a, b, expected) -> None:
assert mul(a, b) == expected
@pytest.mark.parametrize(
"a,b,expected",
[('1', 1, 1)]
)
def test_mul_should_succeed_with_str_params(a, b, expected) -> None:
assert mul(a, b) == expected
When you run Pytest, it will first run the test function with the values in the first element of the list (a = 2, b = 3, expected = 6), and then, it will move on to the second element, the third element, etc., as shown in the output of the pytest -v command.
$ pytest -v =============== test session starts =============== platform darwin -- Python 3.9.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /Users/mikio/pytest/venv/bin/python3 cachedir: .pytest_cache rootdir: /Users/mikio/pytest collected 4 items test_calc.py::test_mul_should_succeed_with_int_params[2-3-6] PASSED [ 25%] test_calc.py::test_mul_should_succeed_with_int_params[5-4-20] PASSED [ 50%] test_calc.py::test_mul_should_succeed_with_int_params[-1-1--1] PASSED [ 75%] test_calc.py::test_mul_should_succeed_with_str_params[1-1-1] PASSED [100%] ================ 4 passed in 0.01s ================
How can I catch exceptions in Pytest?
You can use pytest.raises() as a context manager to verify that the function raises an exception.
The function mul() raises a ValueError if it cannot convert the argument value to an integer.
>>> calc.mul('a', 1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/mikio/pytest/calc.py", line 2, in mul
return int(a) * int(b)
ValueError: invalid literal for int() with base 10: 'a'
You can add a test to verify this behaviour as shown below:
import pytest
from calc import mul
(...)
def test_mul_should_raise_exception_with_non_numeric_str_params() -> None:
with pytest.raises(ValueError):
assert mul('a', 1)
You can run Pytest and check that the test passes.
$ pytest -v =============== test session starts =============== platform darwin -- Python 3.9.1, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /Users/mikio/pytest/venv/bin/python3 cachedir: .pytest_cache rootdir: /Users/mikio/pytest collected 5 items test_calc.py::test_mul_should_succeed_with_int_params[2-3-6] PASSED [ 20%] test_calc.py::test_mul_should_succeed_with_int_params[5-4-20] PASSED [ 40%] test_calc.py::test_mul_should_succeed_with_int_params[-1-1--1] PASSED [ 60%] test_calc.py::test_mul_should_succeed_with_str_params[1-1-1] PASSED [ 80%] test_calc.py::test_mul_should_raise_exception_with_non_numeric_str_params PASSED [100%] ================ 5 passed in 0.01s ================
Summary
In this article, we looked at the popular Python test framework Pytest. First, we looked at how to install Pytest, write a simple test and execute it using the pytest command. We also parametrised the test functions to use various input parameters more efficiently. Then we created a test function to check exceptions.
Pytest is a powerful tool, and this article has only scratched the surface. But even with this basic functionality, you can already write a lot of tests. I hope this article helps you get started with Pytest and improve your code quality.