Files
shopdb-flask/tests/test_plugins/test_endpoints_agent_registration.py
benjamin.baker d32293ccb2 Auto-register staged agent builds as a pushable application
Gate collect_log on a minimum agent version and bridge the agent version
store to the applications table so a staged build can actually be pushed.

update_app resolves an install source from the applications table alone; it
cannot see agentstore. A build could therefore be staged and current while no
application row existed at all, which is what the fleet was in: the agent was
unpushable. Staging is now the single source of truth - it packages the
current build and upserts the row in the same request, so the row cannot
describe a build that is no longer staged.

PARTIAL BUILD ADVISORY
This is steps 1-4 of a longer rollout and is deliberately incomplete:
  - There is no push trigger yet. The registered package is resolvable and
	verifiable, but nothing enqueues an upgrade; the "behind current build"
	indicator is still display-only.
  - The upgrade template script, its script row, and the enqueue route
	(endpoints.agentupdate) are not written.
  - Blocking constraint for the next slice: run_script executes
	powershell.exe -File <script> with NO argument channel, so a delivered
	upgrade script cannot be passed BaseUrl/CollectorKey and must either
	source them itself or run_script must be extended first.
Nothing here touches a live endpoint. Staging only changes what a future
push would contain.

Details:
  - Package members carry a fixed timestamp. zipfile stamps the current mtime
	by default, so identical input produced a different archive digest on
	every call. update_app pins a download by SHA-256, so a drifting hash
	would rewrite installersha256 out from under an endpoint mid-download and
	fail its verification.
  - All three routes that can change 'current' re-register (upload,
	adopt-legacy, rollback). Covering only upload would leave the row pinned
	to the build an operator just rolled away from, and the push would ship
	it anyway - silent drift.
  - Registration cannot fail an upload. The binary is staged and audited by
	the time it runs, so a packaging problem is reported in the response
	payload rather than raised; returning an error would tell the operator a
	successful upload was rejected.
  - The package is written to a temp name and os.replace'd, so an interrupted
	write cannot leave a truncated archive at the path the row advertises.
  - The row is isinstallable but hidden: machine-managed, and it must not
	appear in the software catalogue beside real licensed applications.
  - Added application_package_location() to the shopdb.api contract surface
	(lazy, matching collector_authorized) returning the package directory and
	URL prefix together. agentregistry reached into shopdb.core internals,
	violating the plugin import contract; sourcing both from one place also
	stops the write location and installpath from drifting apart.

collect_log is refused at enqueue below its minimum agent version, with
numeric comparison - string ordering misplaces 1.9.0.0 against 1.16.0. The
refusal is mirrored in the UI so a command slot is not spent on a call that
is guaranteed to fail.

Test isolation: create_app('testing') resolves instance_path to the real repo
instance/ directory, so agent upload tests write into the developer's actual
store. The new tests redirect both the store and package output into tmp_path
and include a guard test that fails if that isolation regresses. NOTE: the
pre-existing test_endpoints_agentupload.py still has this leak and deposits
1.5.0.0 and 1.6.3 into instance/siteagent/ on a full run - not fixed here.

Suite: 2834 passed, 11 skipped. The two failures (test_docs_publishable, and
a symlink scan needing Windows privileges) were confirmed pre-existing by
stashing this work and reproducing them.
2026-09-09 21:40:51 -04:00

251 lines
10 KiB
Python

"""Auto-registration of staged agent builds as a pushable Application.
The endpoint update_app push resolves an install source from the applications
table alone; it cannot see the agent version store. Before this, a build could
be staged and current while no application row existed at all, which is exactly
what was found in the fleet - the agent was unpushable.
These tests pin the properties that make the bridge trustworthy:
- staging creates the row, and re-staging updates it in place (never a row
per build, because appname is unique and drift is the failure mode);
- the packaged archive is byte-stable for unchanged input, because update_app
pins a download to a digest and a drifting hash breaks verification;
- every path that changes 'current' re-registers, including rollback;
- registration failure never fails an upload that already landed on disk.
Uploads use PE-magic bytes with a monkeypatched version parser, matching the
approach in test_endpoints_agentupload.py - building a real PE with an
RT_VERSION resource inline is impractical.
Every test runs against an ISOLATED store. create_app('testing') resolves
instance_path to the real repo instance/ directory, so without redirection
these tests stage fake 130-byte binaries into the developer's actual agent
store and leave invented versions behind in its manifest. The isolate fixture
below is autouse for that reason: it is a correctness requirement, not a
convenience.
"""
import hashlib
import io
import os
import zipfile
import pytest
from plugins.endpoints.api import agentstore, agentupload, agentregistry
from shopdb.core.models.application import Application
UPLOAD_URL = '/api/endpoints/siteagent/binaries/{}'
CURRENT_URL = '/api/endpoints/siteagent/binaries/{}/current'
_PE_BYTES = b'MZ' + b'\x00' * 128
@pytest.fixture(autouse=True)
def isolate_store(tmp_path, monkeypatch):
"""Point the agent store and package output at a per-test directory.
Both must be redirected. Redirecting only the store would still write the
~zip into the real applicationpackages tree under a live application id.
"""
siteagent = tmp_path / 'siteagent'
packages = tmp_path / 'applicationpackages'
siteagent.mkdir()
packages.mkdir()
monkeypatch.setattr(agentstore, '_siteagent_dir', lambda: str(siteagent))
monkeypatch.setattr(agentregistry, 'application_package_location',
lambda: (str(packages), '/api/applications/package/'))
return tmp_path
def _upload(client, headers, monkeypatch, version, binkey='agent',
payload=None, makecurrent=None):
"""Stage a build through the real HTTP route."""
monkeypatch.setattr(agentstore, 'read_pe_file_version', lambda data: version)
form = {'file': (io.BytesIO(payload or _PE_BYTES), 'shopdbagent.exe'),
'version': version}
if makecurrent is not None:
form['makecurrent'] = makecurrent
return client.post(UPLOAD_URL.format(binkey), data=form,
content_type='multipart/form-data', headers=headers)
def _agent_row():
return Application.query.filter_by(
appname=agentregistry.AGENT_APPNAME).first()
def test_staging_a_build_creates_a_pushable_application_row(
client, db, app, auth_headers, monkeypatch):
"""Without this row, update_app has nothing to resolve and cannot push."""
assert _agent_row() is None
response = _upload(client, auth_headers, monkeypatch, '1.16.0.0')
assert response.status_code == 200
registration = response.get_json()['data']['registration']
assert registration['ok'] is True
row = _agent_row()
assert row is not None
# The row must be resolvable and pinned: a path update_app can fetch, and a
# digest it can verify the bytes against.
assert row.installpath == '/api/applications/package/{}/{}'.format(
row.appid, agentregistry.AGENT_PACKAGE_FILENAME)
assert row.installersha256 == registration['sha256']
assert row.isinstallable is True
# Hidden: this is machine-managed, and must not appear in the software
# catalogue beside real licensed applications.
assert row.ishidden is True
def test_restaging_updates_the_same_row_rather_than_adding_one(
client, db, app, auth_headers, monkeypatch):
"""appname is unique; a row per build would break the push target."""
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
first = _agent_row()
firstid = first.appid
response = _upload(client, auth_headers, monkeypatch, '1.17.0.0')
assert response.status_code == 200
assert Application.query.filter_by(
appname=agentregistry.AGENT_APPNAME).count() == 1
row = _agent_row()
assert row.appid == firstid
# The row must follow the newly current build, not keep pointing at 1.16.
assert row.installersha256 == \
response.get_json()['data']['registration']['sha256']
def test_packaging_the_same_build_twice_yields_the_same_digest(
client, db, app, auth_headers, monkeypatch):
"""update_app pins bytes by digest.
zipfile stamps members with the current mtime by default, which made the
archive hash change on every call for identical input. That would rewrite
installersha256 out from under an endpoint mid-download and fail its
SHA-256 verification.
"""
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
firstpayload, firstmanifest = agentstore.build_current_package()
secondpayload, secondmanifest = agentstore.build_current_package()
assert firstmanifest['sha256'] == secondmanifest['sha256']
assert firstpayload == secondpayload
def test_the_package_contains_the_staged_binary_under_a_fixed_name(
client, db, app, auth_headers, monkeypatch):
"""The install script addresses members by name, not by discovery."""
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
payload, manifest = agentstore.build_current_package()
with zipfile.ZipFile(io.BytesIO(payload)) as archive:
assert archive.testzip() is None
names = archive.namelist()
assert agentstore.PACKAGE_MEMBER_NAMES['agent'] in names
assert manifest['agentversion'] == '1.16.0.0'
assert manifest['contents']['agent']['version'] == '1.16.0.0'
def test_the_written_package_file_matches_the_recorded_digest(
client, db, app, auth_headers, monkeypatch):
"""A row whose digest disagrees with the file on disk is unpushable."""
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
row = _agent_row()
path = agentregistry._package_path(row.appid)
assert os.path.isfile(path)
with open(path, 'rb') as handle:
ondisk = hashlib.sha256(handle.read()).hexdigest()
assert ondisk == row.installersha256
# The atomic-write temp file must not survive a successful write.
assert not os.path.isfile(path + '.tmp')
def test_rolling_back_repoints_the_row_at_the_restored_build(
client, db, app, auth_headers, monkeypatch):
"""A rollback changes the bytes the fleet should get.
If only upload re-registered, the row would still pin the build the
operator just rolled away from, and the push would ship it anyway.
"""
_upload(client, auth_headers, monkeypatch, '1.16.0.0',
payload=b'MZ' + b'\x01' * 256)
oldsha = _agent_row().installersha256
_upload(client, auth_headers, monkeypatch, '1.17.0.0',
payload=b'MZ' + b'\x02' * 256)
newsha = _agent_row().installersha256
assert newsha != oldsha
response = client.put(CURRENT_URL.format('agent'),
json={'version': '1.16.0.0'}, headers=auth_headers)
assert response.status_code == 200
assert response.get_json()['data']['registration']['ok'] is True
assert agentstore.current_version('agent') == '1.16.0.0'
# Back to the rolled-to build's package, not the newer one.
assert _agent_row().installersha256 == oldsha
def test_a_registration_failure_does_not_fail_the_upload(
client, db, app, auth_headers, monkeypatch):
"""The binary is already staged and audited by the time this runs.
Turning a packaging problem into an error response would tell the operator
the upload was rejected when it actually succeeded, inviting a re-upload of
a build that is already there.
"""
def _explode():
raise OSError('simulated packaging failure')
monkeypatch.setattr(agentregistry.agentstore, 'build_current_package',
_explode)
response = _upload(client, auth_headers, monkeypatch, '1.16.0.0')
assert response.status_code == 200
body = response.get_json()['data']
# The staging itself still stands...
assert body['current'] == '1.16.0.0'
# ...and the failure is surfaced rather than swallowed.
assert body['registration']['ok'] is False
assert 'simulated packaging failure' in body['registration']['error']
def test_staging_only_a_tray_reports_that_there_is_nothing_to_package(
client, db, app, auth_headers, monkeypatch):
"""The bundle is defined by the agent; a tray alone is not pushable."""
response = _upload(client, auth_headers, monkeypatch, '1.6.5.0',
binkey='tray')
assert response.status_code == 200
registration = response.get_json()['data']['registration']
assert registration['ok'] is False
assert 'no current agent build' in registration['error']
assert _agent_row() is None
def test_these_tests_do_not_touch_the_real_agent_store(
client, db, app, auth_headers, monkeypatch, isolate_store):
"""Guard the isolation itself.
An earlier draft of this file wrote fake 130-byte binaries into the repo's
real instance/siteagent/ tree and invented a version in its manifest,
because create_app('testing') shares the developer's instance_path. If that
redirection ever regresses, fail here rather than silently corrupting a
staged build again.
"""
_upload(client, auth_headers, monkeypatch, '9.9.9.9')
# The staged build landed inside the per-test directory...
staged = isolate_store / 'siteagent' / 'agent' / '9.9.9.9'
assert staged.is_dir()
# ...and the real store never saw it.
realstore = os.path.join(app.instance_path, 'siteagent', 'agent', '9.9.9.9')
assert not os.path.exists(realstore)