Auto-register staged agent builds as a pushable application
Gate collect_log on a minimum agent version and bridge the agent version
store to the applications table so a staged build can actually be pushed.
update_app resolves an install source from the applications table alone; it
cannot see agentstore. A build could therefore be staged and current while no
application row existed at all, which is what the fleet was in: the agent was
unpushable. Staging is now the single source of truth - it packages the
current build and upserts the row in the same request, so the row cannot
describe a build that is no longer staged.
PARTIAL BUILD ADVISORY
This is steps 1-4 of a longer rollout and is deliberately incomplete:
- There is no push trigger yet. The registered package is resolvable and
verifiable, but nothing enqueues an upgrade; the "behind current build"
indicator is still display-only.
- The upgrade template script, its script row, and the enqueue route
(endpoints.agentupdate) are not written.
- Blocking constraint for the next slice: run_script executes
powershell.exe -File <script> with NO argument channel, so a delivered
upgrade script cannot be passed BaseUrl/CollectorKey and must either
source them itself or run_script must be extended first.
Nothing here touches a live endpoint. Staging only changes what a future
push would contain.
Details:
- Package members carry a fixed timestamp. zipfile stamps the current mtime
by default, so identical input produced a different archive digest on
every call. update_app pins a download by SHA-256, so a drifting hash
would rewrite installersha256 out from under an endpoint mid-download and
fail its verification.
- All three routes that can change 'current' re-register (upload,
adopt-legacy, rollback). Covering only upload would leave the row pinned
to the build an operator just rolled away from, and the push would ship
it anyway - silent drift.
- Registration cannot fail an upload. The binary is staged and audited by
the time it runs, so a packaging problem is reported in the response
payload rather than raised; returning an error would tell the operator a
successful upload was rejected.
- The package is written to a temp name and os.replace'd, so an interrupted
write cannot leave a truncated archive at the path the row advertises.
- The row is isinstallable but hidden: machine-managed, and it must not
appear in the software catalogue beside real licensed applications.
- Added application_package_location() to the shopdb.api contract surface
(lazy, matching collector_authorized) returning the package directory and
URL prefix together. agentregistry reached into shopdb.core internals,
violating the plugin import contract; sourcing both from one place also
stops the write location and installpath from drifting apart.
collect_log is refused at enqueue below its minimum agent version, with
numeric comparison - string ordering misplaces 1.9.0.0 against 1.16.0. The
refusal is mirrored in the UI so a command slot is not spent on a call that
is guaranteed to fail.
Test isolation: create_app('testing') resolves instance_path to the real repo
instance/ directory, so agent upload tests write into the developer's actual
store. The new tests redirect both the store and package output into tmp_path
and include a guard test that fails if that isolation regresses. NOTE: the
pre-existing test_endpoints_agentupload.py still has this leak and deposits
1.5.0.0 and 1.6.3 into instance/siteagent/ on a full run - not fixed here.
Suite: 2834 passed, 11 skipped. The two failures (test_docs_publishable, and
a symlink scan needing Windows privileges) were confirmed pre-existing by
stashing this work and reproducing them.
This commit is contained in:
@@ -35,6 +35,7 @@ import json
|
||||
import os
|
||||
import re
|
||||
import struct
|
||||
import zipfile
|
||||
from datetime import datetime
|
||||
|
||||
from flask import current_app
|
||||
@@ -470,3 +471,109 @@ def set_current(binkey, version):
|
||||
manifest['current'] = version
|
||||
_write_manifest(binkey, manifest)
|
||||
return True
|
||||
|
||||
|
||||
def version_sha256(binkey, version):
|
||||
"""The recorded digest for a staged version, or None when not recorded.
|
||||
|
||||
Read from the manifest rather than recomputed: store_version already
|
||||
hashed the exact bytes it wrote, so re-reading the file to hash it again
|
||||
would only introduce a way for the two to disagree.
|
||||
"""
|
||||
for entry in _read_manifest(binkey).get('versions', []):
|
||||
if entry.get('version') == version:
|
||||
return entry.get('sha256')
|
||||
return None
|
||||
|
||||
|
||||
# Name the upgrade payload carries inside the archive. Fixed rather than derived
|
||||
# from the staged filenames so the install script can address its contents
|
||||
# without being told what to expect.
|
||||
PACKAGE_MEMBER_NAMES = {
|
||||
'agent': 'shopdbagent.exe',
|
||||
'tray': 'shopdb-endpoint-tray.exe',
|
||||
}
|
||||
|
||||
# Fixed member timestamp so packaging the same build twice yields byte-identical
|
||||
# archives. The value is arbitrary (the DOS epoch zip supports); only its
|
||||
# constancy matters. Real build provenance lives in the manifest's versions and
|
||||
# digests, not in a zip entry's mtime.
|
||||
_PACKAGE_TIMESTAMP = (1980, 1, 1, 0, 0, 0)
|
||||
|
||||
|
||||
def build_current_package():
|
||||
"""Zip the current agent build (plus the current tray) into one archive.
|
||||
|
||||
Returns (bytes, manifestdict) or (None, None) when no agent build is
|
||||
current. The agent is required; the tray is included only when it too has
|
||||
a current staged version, because an endpoint can legitimately run the
|
||||
service with no tray and half a tray upgrade is worse than none.
|
||||
|
||||
The archive is built IN MEMORY and returned rather than written here: the
|
||||
caller decides where it lands, and nothing under the version store gets a
|
||||
derived artifact that retention would then have to reason about.
|
||||
|
||||
The returned manifest describes what went in - version and digest per
|
||||
member - so the install script can verify it unpacked the build the server
|
||||
meant to send, rather than trusting the archive's filename.
|
||||
"""
|
||||
agentversion = current_version('agent')
|
||||
if not agentversion:
|
||||
return None, None
|
||||
|
||||
members = []
|
||||
agentpath = _version_exe_path('agent', agentversion)
|
||||
if not os.path.isfile(agentpath):
|
||||
return None, None
|
||||
members.append(('agent', agentversion, agentpath))
|
||||
|
||||
trayversion = current_version('tray')
|
||||
if trayversion:
|
||||
traypath = _version_exe_path('tray', trayversion)
|
||||
if os.path.isfile(traypath):
|
||||
members.append(('tray', trayversion, traypath))
|
||||
|
||||
contents = {}
|
||||
buffer = io.BytesIO()
|
||||
# ZIP_DEFLATED, not ZIP_STORED: these are ~70 MB self-contained binaries
|
||||
# carrying the whole .NET runtime, and the archive crosses the wire to
|
||||
# every endpoint being upgraded.
|
||||
with zipfile.ZipFile(buffer, 'w', compression=zipfile.ZIP_DEFLATED) as archive:
|
||||
for binkey, version, path in members:
|
||||
try:
|
||||
with open(path, 'rb') as handle:
|
||||
data = handle.read()
|
||||
except OSError:
|
||||
# A staged file that cannot be read means the package would be
|
||||
# silently incomplete. Refuse the whole thing: a partial upgrade
|
||||
# payload is how an endpoint ends up with a new service exe and
|
||||
# a stale tray.
|
||||
return None, None
|
||||
name = PACKAGE_MEMBER_NAMES[binkey]
|
||||
# Build the entry with a FIXED timestamp. zipfile otherwise stamps
|
||||
# each member with the current local time, which makes the archive
|
||||
# bytes - and therefore its digest - different on every call for
|
||||
# identical input. update_app pins a download to a digest, so a
|
||||
# drifting hash would invalidate a package an endpoint is already
|
||||
# fetching. Same reason the mode is set explicitly.
|
||||
info = zipfile.ZipInfo(name, date_time=_PACKAGE_TIMESTAMP)
|
||||
info.compress_type = zipfile.ZIP_DEFLATED
|
||||
info.external_attr = 0o644 << 16
|
||||
archive.writestr(info, data)
|
||||
contents[binkey] = {
|
||||
'filename': name,
|
||||
'version': version,
|
||||
# Prefer the recorded digest so the package agrees with the
|
||||
# manifest; fall back to hashing when an older manifest entry
|
||||
# predates digest recording.
|
||||
'sha256': (version_sha256(binkey, version)
|
||||
or hashlib.sha256(data).hexdigest()),
|
||||
}
|
||||
|
||||
payload = buffer.getvalue()
|
||||
return payload, {
|
||||
'agentversion': agentversion,
|
||||
'contents': contents,
|
||||
'sha256': hashlib.sha256(payload).hexdigest(),
|
||||
'size': len(payload),
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user