Make the full suite runnable in parallel, and close the dev/prod pin drift

The suite had grown to ~2300 tests and 5m30s. Profiling says there is no hot
spot to fix: the 25 slowest tests sum to about 60s of 331s, and the rest is
per-test fixture cost. The db fixture drops and recreates all 94 tables for
every test, measured at 0.058s a cycle, which is roughly 40% of the wall clock
on its own.

That shape parallelises almost linearly, so pytest-xdist buys more than any
rewrite would:

    pytest -q          2309 passed in 331s
    pytest -q -n auto  2309 passed in 103s     (8 cores)

Deliberately NOT in pytest.ini addopts. Worker startup is dead weight on the
targeted single-file runs that make up most of a working day; the whole-suite
run before a push is where it pays. Opt in with -n auto.

A run distributed across 8 workers also has to be order-independent to pass at
all, so this is incidental evidence that the suite has no inter-test coupling.

Recompiling requirements-dev.txt to add xdist also moved idna, packaging and
pyjwt. That is the -c requirements.txt constraint doing its job rather than a
drift: all three ARE shipped, and the dev file had fallen BEHIND production
(idna 3.13 in dev against 3.19 in prod). CI was testing against older versions
than sites install, which is the exact failure the header comment on that file
was written about. Now matched.

Scope: product
This commit is contained in:
cproudlock
2026-09-02 14:43:00 -04:00
parent 28bf7ea1bf
commit d45a0d15b4
2 changed files with 29 additions and 14 deletions

View File

@@ -10,3 +10,10 @@
pytest>=7.0 pytest>=7.0
pytest-flask>=1.2 pytest-flask>=1.2
pytest-cov>=4.0 pytest-cov>=4.0
# Parallel test execution. The suite is ~2300 tests whose cost is per-test
# fixture teardown (create_all/drop_all is ~0.06s each, ~40% of the wall clock)
# rather than any single slow test, so it parallelises almost linearly: 331s ->
# 103s on 8 cores, same 2309 passing. NOT in pytest.ini addopts - worker startup
# is dead weight on the targeted runs that make up most of a working day. Opt in
# with -n auto when running the whole suite.
pytest-xdist>=3.5

View File

@@ -1,9 +1,5 @@
# This file was autogenerated by uv via the following command: # This file was autogenerated by uv via the following command:
# uv pip compile requirements-dev.in -c requirements.txt --universal --generate-hashes -o requirements-dev.txt # uv pip compile requirements-dev.in -c requirements.txt --universal --generate-hashes -o requirements-dev.txt
#
# The -c constraint pins every shared dependency to the SAME version production
# runs. Without it the two files were compiled at different times and drifted:
# CI tested against alembic 1.18.5 while sites installed 1.18.4.
alembic==1.18.4 \ alembic==1.18.4 \
--hash=sha256:a5ed4adcf6d8a4cb575f3d759f071b03cd6e5c7618eb796cb52497be25bfe19a \ --hash=sha256:a5ed4adcf6d8a4cb575f3d759f071b03cd6e5c7618eb796cb52497be25bfe19a \
--hash=sha256:cb6e1fd84b6174ab8dbb2329f86d631ba9559dd78df550b57804d607672cedbc --hash=sha256:cb6e1fd84b6174ab8dbb2329f86d631ba9559dd78df550b57804d607672cedbc
@@ -434,6 +430,10 @@ email-validator==2.3.0 \
# via # via
# -c requirements.txt # -c requirements.txt
# -r requirements.in # -r requirements.in
execnet==2.1.2 \
--hash=sha256:63d83bfdd9a23e35b9c6a3261412324f964c2ec8dcd8d3c6916ee9373e0befcd \
--hash=sha256:67fba928dd5a544b783f6056f449e5e3931a5c378b128bc18501f7ea79e296ec
# via pytest-xdist
flask==3.1.3 \ flask==3.1.3 \
--hash=sha256:0ef0e52b8a9cd932855379197dd8f94047b359ca0a78695144304cb45f87c9eb \ --hash=sha256:0ef0e52b8a9cd932855379197dd8f94047b359ca0a78695144304cb45f87c9eb \
--hash=sha256:f4bcbefc124291925f1a26446da31a5178f9483862233b23c0c96a20701f670c --hash=sha256:f4bcbefc124291925f1a26446da31a5178f9483862233b23c0c96a20701f670c
@@ -547,9 +547,9 @@ greenlet==3.5.0 ; platform_machine == 'AMD64' or platform_machine == 'WIN32' or
# via # via
# -c requirements.txt # -c requirements.txt
# sqlalchemy # sqlalchemy
idna==3.13 \ idna==3.19 \
--hash=sha256:585ea8fe5d69b9181ec1afba340451fba6ba764af97026f92a91d4eef164a242 \ --hash=sha256:5e0811a4383b21dc5838069f801c4fb62113b7447663d2530d2bd6e77b49bf15 \
--hash=sha256:892ea0cde124a99ce773decba204c5552b69c3c67ffd5f232eb7696135bc8bb3 --hash=sha256:815e7be7a7806d54abb586dc943addc79e8b2ee16915059658cbeff4b1b43bf4
# via # via
# -c requirements.txt # -c requirements.txt
# email-validator # email-validator
@@ -715,10 +715,13 @@ mysql-connector-python==9.7.0 \
# via # via
# -c requirements.txt # -c requirements.txt
# -r requirements.in # -r requirements.in
packaging==26.2 \ packaging==26.3 \
--hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ --hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \
--hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c
# via pytest # via
# -c requirements.txt
# -r requirements.in
# pytest
pluggy==1.6.0 \ pluggy==1.6.0 \
--hash=sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3 \ --hash=sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3 \
--hash=sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746 --hash=sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746
@@ -735,9 +738,9 @@ pygments==2.20.0 \
--hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \
--hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176
# via pytest # via pytest
pyjwt==2.12.1 \ pyjwt==2.13.0 \
--hash=sha256:28ca37c070cad8ba8cd9790cd940535d40274d22f80ab87f3ac6a713e6e8454c \ --hash=sha256:41571c89ca91598c79e8ef18a2d07367d4810fbbd6f637794879baf1b7703423 \
--hash=sha256:c74a7a2adf861c04d002db713dd85f84beb242228e671280bf709d765b03672b --hash=sha256:66adcc2aff09b3f1bbd95fc1e1577df8ac8723c978552fd43304c8a290ac5728
# via # via
# -c requirements.txt # -c requirements.txt
# flask-jwt-extended # flask-jwt-extended
@@ -754,6 +757,7 @@ pytest==9.1.1 \
# -r requirements-dev.in # -r requirements-dev.in
# pytest-cov # pytest-cov
# pytest-flask # pytest-flask
# pytest-xdist
pytest-cov==7.1.0 \ pytest-cov==7.1.0 \
--hash=sha256:30674f2b5f6351aa09702a9c8c364f6a01c27aae0c1366ae8016160d1efc56b2 \ --hash=sha256:30674f2b5f6351aa09702a9c8c364f6a01c27aae0c1366ae8016160d1efc56b2 \
--hash=sha256:a0461110b7865f9a271aa1b51e516c9a95de9d696734a2f71e3e78f46e1d4678 --hash=sha256:a0461110b7865f9a271aa1b51e516c9a95de9d696734a2f71e3e78f46e1d4678
@@ -762,6 +766,10 @@ pytest-flask==1.3.0 \
--hash=sha256:58be1c97b21ba3c4d47e0a7691eb41007748506c36bf51004f78df10691fa95e \ --hash=sha256:58be1c97b21ba3c4d47e0a7691eb41007748506c36bf51004f78df10691fa95e \
--hash=sha256:c0e36e6b0fddc3b91c4362661db83fa694d1feb91fa505475be6732b5bc8c253 --hash=sha256:c0e36e6b0fddc3b91c4362661db83fa694d1feb91fa505475be6732b5bc8c253
# via -r requirements-dev.in # via -r requirements-dev.in
pytest-xdist==3.8.0 \
--hash=sha256:202ca578cfeb7370784a8c33d6d05bc6e13b4f25b5053c30a152269fd10f0b88 \
--hash=sha256:7e578125ec9bc6050861aa93f2d59f1d8d085595d6551c2c90b6f4fad8d3a9f1
# via -r requirements-dev.in
python-dotenv==1.2.2 \ python-dotenv==1.2.2 \
--hash=sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a \ --hash=sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a \
--hash=sha256:2c371a91fbd7ba082c2c1dc1f8bf89ca22564a087c2c287cd9b662adde799cf3 --hash=sha256:2c371a91fbd7ba082c2c1dc1f8bf89ca22564a087c2c287cd9b662adde799cf3