In web development it's common to define a serialization layer that an API/view layer can use to convert objects fetched from the database into e.g. a JSON representation fit for transport over HTTP. With Django REST framework, these Serializers can be nested and mixed to allow reusing code that defines which fields to use from a given resource and how to construct them.
As with many other constructs in Django, this can easily lead to lazily triggered database queries when no care is taken to tell the framework to select related fields aka do a JOIN of the associated tables beforehand. As Serializers evolve over time, performance often degrades quietly, without anyone noticing. How can we test for things like this?
Writing a test that does some request-response and involves serialization is a start, but how to decide if things are "fast enough". Time taken for a test can vary heavily based on the hardware it runs on and what other things the machine is doing, state of network etc. we could only do very broad sanity checks like "this test should not take longer than one minute" and that would not be very helpful to detect real world performance degradation.
One thing I find increasingly helpful is to instead test for the number of database queries made by the system while the test runs or within specific blocks in it. Now, a low number of complex queries may actually be slower than a higher number of simple queries, so the number itself does not exactly mean things are fast. But from my experience, when things go wrong in the future, that's often due to newly introduced lazy database queries, e.g. by including some nested Serializer without adapting the associated QuerySet to prefetch correctly.
pytest-django comes with two neat fixtures for this already, django_assert_num_queries and django_assert_max_num_queries. They can be used as contextmanagers around the piece of code that you care about. I usually start like this:
with django_assert_num_queries(0):
some_call_that_talks_to_the_database()That will fail, since there will be more than zero database queries. It will write the current number to the console. I can then update the test accordingly. From then on, this can be used to detect changes in the number of queries, in both directions, so you will also be notified by your failing test when things get "better" . This is good in my opinion, because less queries can also be a sign of accidental change instead of optimization, e.g. someone deleted or invalidated a filter on a QuerySet.
If you don't care about the query count going down, or the number of queries may be dynamic and you only care about an upper limit, django_assert_max_num_queries would be the better choice.
If the count for some reason fluctuates and you want to check that it stays within some tolerable range, you can use my django_assert_num_between_queries:
from contextlib import contextmanager
from datetime import timedelta
from functools import partial
from typing import Generator
@contextmanager
def _assert_between_num_queries(
config,
min_num: int,
max_num: int,
connection=None,
info=None,
) -> Generator["django.test.utils.CaptureQueriesContext", None, None]:
"""Based on pytest-django's internal _assert_num_queries() helper.
See:
https://github.com/pytest-dev/pytest-django/blob/4e125e16644a3f88a8f8f9e34f9528c7b0a7abea/pytest_django/fixtures.py#L501-L538
"""
from django.test.utils import CaptureQueriesContext
if connection is None:
from django.db import connection as conn
else:
conn = connection
verbose = config.getoption("verbose") > 0
with CaptureQueriesContext(conn) as context:
yield context
num_performed = len(context)
failed = num_performed not in range(min_num, max_num + 1)
if failed:
msg = "Expected to perform between {} and {} queries {}".format(
min_num,
max_num,
"but {} done".format(
num_performed == 1 and "1 was" or "{} were".format(num_performed)
),
)
if info:
msg += "\n{}".format(info)
if verbose:
sqls = (q["sql"] for q in context.captured_queries)
msg += "\n\nQueries:\n========\n\n" + "\n\n".join(sqls)
else:
msg += " (add -v option to show queries)"
pytest.fail(msg)
@pytest.fixture(scope="function")
def django_assert_between_num_queries(pytestconfig):
"""Assert queries lie in specified min/max range.
Use like:
with django_assert_between_num_queries(20, 21):
...
"""
return partial(_assert_between_num_queries, pytestconfig)This "technique" has saved me a few times already, when some innocent little change suddenly affected query performance in a large part of our application and only the tests made that obvious. It has also proven very helpful to document past performance of certain endpoints and can act as a baseline to verify that an optimization worked as expected.