Stop asking for more rows than the server will ever return

You are right that this is the one we keep hitting. perpage is capped at 100
server-side, and the cap is applied SILENTLY: a request for 500 returns the
first 100 rows with a 200, so a picker built that way shows the first 100
options and drops the rest with nothing anywhere reporting a problem. The
missing vendor simply is not in the list.

Twelve call sites were still doing it - vendor and business-unit pickers on the
machine and measuring-tool forms, and the type lists behind PCs, machines,
printers, network devices, measuring tools and configuration policies. Each now
either pages through with listAll()/fetchAllPages() or asks for what it can
actually have. vendorsApi and businessunitsApi gained listAll(), which they
lacked, which is why those two sites reached for 500 in the first place.

The gate is the point. tests/test_core/test_perpage_cap.py reads every .vue and
.js source under frontend/src and plugins/*/frontend and fails on any perpage
over 100. It has to read the SOURCE rather than the running app, because the
response is a valid 200 either way - no runtime assertion can see this one.
Verified by planting a perpage: 500 back into MachineForm: the gate names that
exact line.

Separately, /api/auditlogs built its own meta with the pagination fields flat
and a 'pages' key, while every other list returns
meta.pagination.{page,perpage,total,totalpages,hasnext,hasprev}. A client
written against the documented contract found nothing where it looked, on the
one list whose whole purpose is being read by somebody checking what happened.
It uses paginated_response now.

Also carried here, from the previous batch's working tree: Asset.to_dict emits
ownlocationid/ownmapx/ownmapy/ownlevelid alongside the resolved values, and the
PC form binds to those. The merge of an inherited location is right for a reader
and wrong for a form, which could not tell an owned value from a derived one and
wrote the inherited one back as the record's own.

Scope: product
This commit is contained in:
cproudlock
2026-09-03 16:49:55 -04:00
parent 81ad221e6a
commit 30d0d4f6d5
12 changed files with 109 additions and 22 deletions

View File

@@ -0,0 +1,79 @@
"""No client may ask for a page bigger than the server will give.
MAX_PAGE_SIZE caps perpage at 100. The cap is deliberate, and it is applied
SILENTLY: a request for perpage=500 returns the first 100 rows with a 200, so
nothing anywhere reports a problem. A picker built that way shows the first 100
options and drops the rest, and the person using it has no way to tell - the
missing vendor simply is not in the list.
That shape has been found and fixed more than once, which is what makes it worth
a gate rather than a comment. Paging through is what fetchAllPages/listAll are
for.
The check reads the frontend sources rather than the running app, because the
defect is in what the caller ASKS for: the response is a valid 200 either way,
so no runtime assertion can see it.
"""
import re
from pathlib import Path
REPO = Path(__file__).resolve().parents[2]
# Where a client is written. The staged copy under frontend/src/.plugins-staged
# is generated from plugins/*/frontend at build time, so checking the source
# avoids reporting the same line twice.
SOURCE_DIRS = [
REPO / 'frontend' / 'src',
*sorted((REPO / 'plugins').glob('*/frontend')),
]
SKIP_PARTS = {'.plugins-staged', 'node_modules', 'dist'}
# perpage: <number>, in an object literal or a query string.
PERPAGE = re.compile(r'perpage[\'"]?\s*[:=]\s*[\'"]?(\d+)', re.IGNORECASE)
MAX_PAGE_SIZE = 100
def _sources():
for root in SOURCE_DIRS:
if not root.exists():
continue
for path in root.rglob('*'):
if path.suffix not in ('.vue', '.js'):
continue
if set(path.parts) & SKIP_PARTS:
continue
if path.name.endswith('.spec.js'):
continue
yield path
def test_the_sweep_actually_reads_something():
"""A gate that matches nothing passes forever."""
files = list(_sources())
assert len(files) > 100, len(files)
assert any(PERPAGE.search(path.read_text()) for path in files), \
'no perpage usage found at all - the pattern is probably wrong'
def test_no_client_asks_for_more_rows_than_the_server_returns():
offenders = []
for path in _sources():
for lineno, line in enumerate(path.read_text().splitlines(), start=1):
# A comment explaining the cap is not a caller asking for 1000
# rows. Crude but sufficient: this is a line-oriented check over
# source that follows one style.
stripped = line.strip()
if stripped.startswith('//') or stripped.startswith('*'):
continue
for match in PERPAGE.finditer(line):
if int(match.group(1)) > MAX_PAGE_SIZE:
offenders.append('%s:%d %s'
% (path.relative_to(REPO), lineno,
line.strip()[:90]))
assert not offenders, (
'perpage is capped at %d server-side and the excess is dropped '
'silently, so these show a truncated list with no error. Use listAll() '
'/ fetchAllPages() instead:\n %s'
% (MAX_PAGE_SIZE, '\n '.join(offenders)))