Gate collect_log on a minimum agent version and bridge the agent version
store to the applications table so a staged build can actually be pushed.
update_app resolves an install source from the applications table alone; it
cannot see agentstore. A build could therefore be staged and current while no
application row existed at all, which is what the fleet was in: the agent was
unpushable. Staging is now the single source of truth - it packages the
current build and upserts the row in the same request, so the row cannot
describe a build that is no longer staged.
PARTIAL BUILD ADVISORY
This is steps 1-4 of a longer rollout and is deliberately incomplete:
- There is no push trigger yet. The registered package is resolvable and
verifiable, but nothing enqueues an upgrade; the "behind current build"
indicator is still display-only.
- The upgrade template script, its script row, and the enqueue route
(endpoints.agentupdate) are not written.
- Blocking constraint for the next slice: run_script executes
powershell.exe -File <script> with NO argument channel, so a delivered
upgrade script cannot be passed BaseUrl/CollectorKey and must either
source them itself or run_script must be extended first.
Nothing here touches a live endpoint. Staging only changes what a future
push would contain.
Details:
- Package members carry a fixed timestamp. zipfile stamps the current mtime
by default, so identical input produced a different archive digest on
every call. update_app pins a download by SHA-256, so a drifting hash
would rewrite installersha256 out from under an endpoint mid-download and
fail its verification.
- All three routes that can change 'current' re-register (upload,
adopt-legacy, rollback). Covering only upload would leave the row pinned
to the build an operator just rolled away from, and the push would ship
it anyway - silent drift.
- Registration cannot fail an upload. The binary is staged and audited by
the time it runs, so a packaging problem is reported in the response
payload rather than raised; returning an error would tell the operator a
successful upload was rejected.
- The package is written to a temp name and os.replace'd, so an interrupted
write cannot leave a truncated archive at the path the row advertises.
- The row is isinstallable but hidden: machine-managed, and it must not
appear in the software catalogue beside real licensed applications.
- Added application_package_location() to the shopdb.api contract surface
(lazy, matching collector_authorized) returning the package directory and
URL prefix together. agentregistry reached into shopdb.core internals,
violating the plugin import contract; sourcing both from one place also
stops the write location and installpath from drifting apart.
collect_log is refused at enqueue below its minimum agent version, with
numeric comparison - string ordering misplaces 1.9.0.0 against 1.16.0. The
refusal is mirrored in the UI so a command slot is not spent on a call that
is guaranteed to fail.
Test isolation: create_app('testing') resolves instance_path to the real repo
instance/ directory, so agent upload tests write into the developer's actual
store. The new tests redirect both the store and package output into tmp_path
and include a guard test that fails if that isolation regresses. NOTE: the
pre-existing test_endpoints_agentupload.py still has this leak and deposits
1.5.0.0 and 1.6.3 into instance/siteagent/ on a full run - not fixed here.
Suite: 2834 passed, 11 skipped. The two failures (test_docs_publishable, and
a symlink scan needing Windows privileges) were confirmed pre-existing by
stashing this work and reproducing them.
251 lines
10 KiB
Python
251 lines
10 KiB
Python
"""Auto-registration of staged agent builds as a pushable Application.
|
|
|
|
The endpoint update_app push resolves an install source from the applications
|
|
table alone; it cannot see the agent version store. Before this, a build could
|
|
be staged and current while no application row existed at all, which is exactly
|
|
what was found in the fleet - the agent was unpushable.
|
|
|
|
These tests pin the properties that make the bridge trustworthy:
|
|
- staging creates the row, and re-staging updates it in place (never a row
|
|
per build, because appname is unique and drift is the failure mode);
|
|
- the packaged archive is byte-stable for unchanged input, because update_app
|
|
pins a download to a digest and a drifting hash breaks verification;
|
|
- every path that changes 'current' re-registers, including rollback;
|
|
- registration failure never fails an upload that already landed on disk.
|
|
|
|
Uploads use PE-magic bytes with a monkeypatched version parser, matching the
|
|
approach in test_endpoints_agentupload.py - building a real PE with an
|
|
RT_VERSION resource inline is impractical.
|
|
|
|
Every test runs against an ISOLATED store. create_app('testing') resolves
|
|
instance_path to the real repo instance/ directory, so without redirection
|
|
these tests stage fake 130-byte binaries into the developer's actual agent
|
|
store and leave invented versions behind in its manifest. The isolate fixture
|
|
below is autouse for that reason: it is a correctness requirement, not a
|
|
convenience.
|
|
"""
|
|
|
|
import hashlib
|
|
import io
|
|
import os
|
|
import zipfile
|
|
|
|
import pytest
|
|
|
|
from plugins.endpoints.api import agentstore, agentupload, agentregistry
|
|
from shopdb.core.models.application import Application
|
|
|
|
UPLOAD_URL = '/api/endpoints/siteagent/binaries/{}'
|
|
CURRENT_URL = '/api/endpoints/siteagent/binaries/{}/current'
|
|
|
|
_PE_BYTES = b'MZ' + b'\x00' * 128
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def isolate_store(tmp_path, monkeypatch):
|
|
"""Point the agent store and package output at a per-test directory.
|
|
|
|
Both must be redirected. Redirecting only the store would still write the
|
|
~zip into the real applicationpackages tree under a live application id.
|
|
"""
|
|
siteagent = tmp_path / 'siteagent'
|
|
packages = tmp_path / 'applicationpackages'
|
|
siteagent.mkdir()
|
|
packages.mkdir()
|
|
monkeypatch.setattr(agentstore, '_siteagent_dir', lambda: str(siteagent))
|
|
monkeypatch.setattr(agentregistry, 'application_package_location',
|
|
lambda: (str(packages), '/api/applications/package/'))
|
|
return tmp_path
|
|
|
|
|
|
def _upload(client, headers, monkeypatch, version, binkey='agent',
|
|
payload=None, makecurrent=None):
|
|
"""Stage a build through the real HTTP route."""
|
|
monkeypatch.setattr(agentstore, 'read_pe_file_version', lambda data: version)
|
|
form = {'file': (io.BytesIO(payload or _PE_BYTES), 'shopdbagent.exe'),
|
|
'version': version}
|
|
if makecurrent is not None:
|
|
form['makecurrent'] = makecurrent
|
|
return client.post(UPLOAD_URL.format(binkey), data=form,
|
|
content_type='multipart/form-data', headers=headers)
|
|
|
|
|
|
def _agent_row():
|
|
return Application.query.filter_by(
|
|
appname=agentregistry.AGENT_APPNAME).first()
|
|
|
|
|
|
def test_staging_a_build_creates_a_pushable_application_row(
|
|
client, db, app, auth_headers, monkeypatch):
|
|
"""Without this row, update_app has nothing to resolve and cannot push."""
|
|
assert _agent_row() is None
|
|
|
|
response = _upload(client, auth_headers, monkeypatch, '1.16.0.0')
|
|
assert response.status_code == 200
|
|
|
|
registration = response.get_json()['data']['registration']
|
|
assert registration['ok'] is True
|
|
|
|
row = _agent_row()
|
|
assert row is not None
|
|
# The row must be resolvable and pinned: a path update_app can fetch, and a
|
|
# digest it can verify the bytes against.
|
|
assert row.installpath == '/api/applications/package/{}/{}'.format(
|
|
row.appid, agentregistry.AGENT_PACKAGE_FILENAME)
|
|
assert row.installersha256 == registration['sha256']
|
|
assert row.isinstallable is True
|
|
# Hidden: this is machine-managed, and must not appear in the software
|
|
# catalogue beside real licensed applications.
|
|
assert row.ishidden is True
|
|
|
|
|
|
def test_restaging_updates_the_same_row_rather_than_adding_one(
|
|
client, db, app, auth_headers, monkeypatch):
|
|
"""appname is unique; a row per build would break the push target."""
|
|
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
|
|
first = _agent_row()
|
|
firstid = first.appid
|
|
|
|
response = _upload(client, auth_headers, monkeypatch, '1.17.0.0')
|
|
assert response.status_code == 200
|
|
|
|
assert Application.query.filter_by(
|
|
appname=agentregistry.AGENT_APPNAME).count() == 1
|
|
row = _agent_row()
|
|
assert row.appid == firstid
|
|
# The row must follow the newly current build, not keep pointing at 1.16.
|
|
assert row.installersha256 == \
|
|
response.get_json()['data']['registration']['sha256']
|
|
|
|
|
|
def test_packaging_the_same_build_twice_yields_the_same_digest(
|
|
client, db, app, auth_headers, monkeypatch):
|
|
"""update_app pins bytes by digest.
|
|
|
|
zipfile stamps members with the current mtime by default, which made the
|
|
archive hash change on every call for identical input. That would rewrite
|
|
installersha256 out from under an endpoint mid-download and fail its
|
|
SHA-256 verification.
|
|
"""
|
|
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
|
|
|
|
firstpayload, firstmanifest = agentstore.build_current_package()
|
|
secondpayload, secondmanifest = agentstore.build_current_package()
|
|
|
|
assert firstmanifest['sha256'] == secondmanifest['sha256']
|
|
assert firstpayload == secondpayload
|
|
|
|
|
|
def test_the_package_contains_the_staged_binary_under_a_fixed_name(
|
|
client, db, app, auth_headers, monkeypatch):
|
|
"""The install script addresses members by name, not by discovery."""
|
|
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
|
|
|
|
payload, manifest = agentstore.build_current_package()
|
|
with zipfile.ZipFile(io.BytesIO(payload)) as archive:
|
|
assert archive.testzip() is None
|
|
names = archive.namelist()
|
|
assert agentstore.PACKAGE_MEMBER_NAMES['agent'] in names
|
|
assert manifest['agentversion'] == '1.16.0.0'
|
|
assert manifest['contents']['agent']['version'] == '1.16.0.0'
|
|
|
|
|
|
def test_the_written_package_file_matches_the_recorded_digest(
|
|
client, db, app, auth_headers, monkeypatch):
|
|
"""A row whose digest disagrees with the file on disk is unpushable."""
|
|
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
|
|
row = _agent_row()
|
|
|
|
path = agentregistry._package_path(row.appid)
|
|
assert os.path.isfile(path)
|
|
with open(path, 'rb') as handle:
|
|
ondisk = hashlib.sha256(handle.read()).hexdigest()
|
|
assert ondisk == row.installersha256
|
|
# The atomic-write temp file must not survive a successful write.
|
|
assert not os.path.isfile(path + '.tmp')
|
|
|
|
|
|
def test_rolling_back_repoints_the_row_at_the_restored_build(
|
|
client, db, app, auth_headers, monkeypatch):
|
|
"""A rollback changes the bytes the fleet should get.
|
|
|
|
If only upload re-registered, the row would still pin the build the
|
|
operator just rolled away from, and the push would ship it anyway.
|
|
"""
|
|
_upload(client, auth_headers, monkeypatch, '1.16.0.0',
|
|
payload=b'MZ' + b'\x01' * 256)
|
|
oldsha = _agent_row().installersha256
|
|
|
|
_upload(client, auth_headers, monkeypatch, '1.17.0.0',
|
|
payload=b'MZ' + b'\x02' * 256)
|
|
newsha = _agent_row().installersha256
|
|
assert newsha != oldsha
|
|
|
|
response = client.put(CURRENT_URL.format('agent'),
|
|
json={'version': '1.16.0.0'}, headers=auth_headers)
|
|
assert response.status_code == 200
|
|
assert response.get_json()['data']['registration']['ok'] is True
|
|
|
|
assert agentstore.current_version('agent') == '1.16.0.0'
|
|
# Back to the rolled-to build's package, not the newer one.
|
|
assert _agent_row().installersha256 == oldsha
|
|
|
|
|
|
def test_a_registration_failure_does_not_fail_the_upload(
|
|
client, db, app, auth_headers, monkeypatch):
|
|
"""The binary is already staged and audited by the time this runs.
|
|
|
|
Turning a packaging problem into an error response would tell the operator
|
|
the upload was rejected when it actually succeeded, inviting a re-upload of
|
|
a build that is already there.
|
|
"""
|
|
def _explode():
|
|
raise OSError('simulated packaging failure')
|
|
|
|
monkeypatch.setattr(agentregistry.agentstore, 'build_current_package',
|
|
_explode)
|
|
|
|
response = _upload(client, auth_headers, monkeypatch, '1.16.0.0')
|
|
|
|
assert response.status_code == 200
|
|
body = response.get_json()['data']
|
|
# The staging itself still stands...
|
|
assert body['current'] == '1.16.0.0'
|
|
# ...and the failure is surfaced rather than swallowed.
|
|
assert body['registration']['ok'] is False
|
|
assert 'simulated packaging failure' in body['registration']['error']
|
|
|
|
|
|
def test_staging_only_a_tray_reports_that_there_is_nothing_to_package(
|
|
client, db, app, auth_headers, monkeypatch):
|
|
"""The bundle is defined by the agent; a tray alone is not pushable."""
|
|
response = _upload(client, auth_headers, monkeypatch, '1.6.5.0',
|
|
binkey='tray')
|
|
|
|
assert response.status_code == 200
|
|
registration = response.get_json()['data']['registration']
|
|
assert registration['ok'] is False
|
|
assert 'no current agent build' in registration['error']
|
|
assert _agent_row() is None
|
|
|
|
|
|
def test_these_tests_do_not_touch_the_real_agent_store(
|
|
client, db, app, auth_headers, monkeypatch, isolate_store):
|
|
"""Guard the isolation itself.
|
|
|
|
An earlier draft of this file wrote fake 130-byte binaries into the repo's
|
|
real instance/siteagent/ tree and invented a version in its manifest,
|
|
because create_app('testing') shares the developer's instance_path. If that
|
|
redirection ever regresses, fail here rather than silently corrupting a
|
|
staged build again.
|
|
"""
|
|
_upload(client, auth_headers, monkeypatch, '9.9.9.9')
|
|
|
|
# The staged build landed inside the per-test directory...
|
|
staged = isolate_store / 'siteagent' / 'agent' / '9.9.9.9'
|
|
assert staged.is_dir()
|
|
|
|
# ...and the real store never saw it.
|
|
realstore = os.path.join(app.instance_path, 'siteagent', 'agent', '9.9.9.9')
|
|
assert not os.path.exists(realstore)
|