Auto-register staged agent builds as a pushable application

Gate collect_log on a minimum agent version and bridge the agent version
store to the applications table so a staged build can actually be pushed.

update_app resolves an install source from the applications table alone; it
cannot see agentstore. A build could therefore be staged and current while no
application row existed at all, which is what the fleet was in: the agent was
unpushable. Staging is now the single source of truth - it packages the
current build and upserts the row in the same request, so the row cannot
describe a build that is no longer staged.

PARTIAL BUILD ADVISORY
This is steps 1-4 of a longer rollout and is deliberately incomplete:
  - There is no push trigger yet. The registered package is resolvable and
	verifiable, but nothing enqueues an upgrade; the "behind current build"
	indicator is still display-only.
  - The upgrade template script, its script row, and the enqueue route
	(endpoints.agentupdate) are not written.
  - Blocking constraint for the next slice: run_script executes
	powershell.exe -File <script> with NO argument channel, so a delivered
	upgrade script cannot be passed BaseUrl/CollectorKey and must either
	source them itself or run_script must be extended first.
Nothing here touches a live endpoint. Staging only changes what a future
push would contain.

Details:
  - Package members carry a fixed timestamp. zipfile stamps the current mtime
	by default, so identical input produced a different archive digest on
	every call. update_app pins a download by SHA-256, so a drifting hash
	would rewrite installersha256 out from under an endpoint mid-download and
	fail its verification.
  - All three routes that can change 'current' re-register (upload,
	adopt-legacy, rollback). Covering only upload would leave the row pinned
	to the build an operator just rolled away from, and the push would ship
	it anyway - silent drift.
  - Registration cannot fail an upload. The binary is staged and audited by
	the time it runs, so a packaging problem is reported in the response
	payload rather than raised; returning an error would tell the operator a
	successful upload was rejected.
  - The package is written to a temp name and os.replace'd, so an interrupted
	write cannot leave a truncated archive at the path the row advertises.
  - The row is isinstallable but hidden: machine-managed, and it must not
	appear in the software catalogue beside real licensed applications.
  - Added application_package_location() to the shopdb.api contract surface
	(lazy, matching collector_authorized) returning the package directory and
	URL prefix together. agentregistry reached into shopdb.core internals,
	violating the plugin import contract; sourcing both from one place also
	stops the write location and installpath from drifting apart.

collect_log is refused at enqueue below its minimum agent version, with
numeric comparison - string ordering misplaces 1.9.0.0 against 1.16.0. The
refusal is mirrored in the UI so a command slot is not spent on a call that
is guaranteed to fail.

Test isolation: create_app('testing') resolves instance_path to the real repo
instance/ directory, so agent upload tests write into the developer's actual
store. The new tests redirect both the store and package output into tmp_path
and include a guard test that fails if that isolation regresses. NOTE: the
pre-existing test_endpoints_agentupload.py still has this leak and deposits
1.5.0.0 and 1.6.3 into instance/siteagent/ on a full run - not fixed here.

Suite: 2834 passed, 11 skipped. The two failures (test_docs_publishable, and
a symlink scan needing Windows privileges) were confirmed pre-existing by
stashing this work and reproducing them.
This commit is contained in:
benjamin.baker
2026-09-09 21:40:51 -04:00
parent eee06743a6
commit d32293ccb2
11 changed files with 983 additions and 6 deletions

View File

@@ -695,6 +695,15 @@
again. Previously collected logs are still readable below.
</div>
<!-- Capability, not permission. Shown even while the endpoint is
offline, because "too old" stays true when it comes back and an
operator planning work should see both facts at once. History
below still reads: logs collected before are unaffected. -->
<div
v-if="!agentSupportsLogCollection"
class="logcollect-note logcollect-note--warn"
>{{ logCollectorVersionNotice }}</div>
<div class="logcollect-form">
<label class="logcollect-label" for="logcollect-path">
Full path to the log file
@@ -705,7 +714,7 @@
class="logcollect-input mono"
type="text"
placeholder="C:\ProgramData\ShopDB\EndpointAgent\agent.log"
:disabled="collectingLog || !liveness"
:disabled="collectingLog || !liveness || !agentSupportsLogCollection"
@keyup.enter="collectLog"
/>
<p v-if="logpatherror" class="logcollect-error">{{ logpatherror }}</p>
@@ -719,7 +728,7 @@
<div class="logcollect-actions">
<button
class="btn btn-warning"
:disabled="!logpath.trim() || collectingLog || !liveness"
:disabled="!logpath.trim() || collectingLog || !liveness || !agentSupportsLogCollection"
@click="collectLog"
>{{ collectingLog ? 'Collecting...' : 'Collect Log' }}</button>
</div>
@@ -1031,6 +1040,15 @@ const computer = ref(null)
// when the viewer lacks endpoints.view - in either case the chip is simply not
// shown, which is better than rendering "unknown" and implying a fault.
const liveness = ref(null)
// Agent version this endpoint last reported, for capability gating. Null when
// the endpoints plugin is absent, the viewer cannot read telemetry, or the
// endpoint has never checked in.
const agentversion = ref(null)
// The currently published agent build, and the per-action version floors, both
// served by the endpoint state route. Held rather than hardcoded so raising a
// floor on the server cannot leave this page disagreeing with the API.
const agentcurrent = ref(null)
const actionminimums = ref({})
const installedApps = ref([])
const assetPanels = ref(null)
@@ -1459,10 +1477,19 @@ async function loadLiveness(assetid) {
if (state.status) {
liveness.value = { level: levels[state.status], label: labels[state.status] }
}
// Already in this payload, so no second request. Used to tell an operator
// that an action needs a newer agent BEFORE they queue a command that the
// agent will refuse with a message that reads like a permissions problem.
agentversion.value = state.agentversion || null
agentcurrent.value = state.agentcurrent || null
actionminimums.value = state.actionminimums || {}
} catch (error) {
// 404 = no agent on this PC, 403 = the viewer cannot read telemetry.
// Neither is a page error; the chip just does not appear.
liveness.value = null
agentversion.value = null
agentcurrent.value = null
actionminimums.value = {}
}
}
@@ -1798,6 +1825,62 @@ const logcollectionsloading = ref(false)
const activelog = ref(null)
const logcontentloading = ref(false)
// Log collection needs an agent new enough to have the action. An older agent
// has no case for it and answers "Unknown or disallowed action", which reads as
// a permissions refusal - so an operator checks a grant they already hold
// rather than the agent version that is actually the problem. The server
// refuses these too; this only stops the pointless round trip and says why.
const collectLogMinAgent = computed(() => actionminimums.value.collect_log || null)
// Numeric per part, never lexical. Agents report four parts ('1.15.0.0') while
// the build carries three ('1.16.0'), and a string compare would also rank
// '1.9.0.0' above '1.16.0' because '9' > '1'. Mirrors agent_meets_minimum on
// the server; both must agree or the button and the API disagree about the
// same endpoint.
function parseAgentVersion(value) {
if (!value) return null
const parts = String(value).trim().split('.')
const out = []
for (const chunk of parts) {
if (!/^\d+$/.test(chunk)) return null
out.push(Number(chunk))
}
if (!out.length) return null
while (out.length < 4) out.push(0)
return out.slice(0, 4)
}
function agentMeetsMinimum(reported, minimum) {
const reportedparts = parseAgentVersion(reported)
const minimumparts = parseAgentVersion(minimum)
// Fail closed on an unreadable version. Seeded rows carry '0.0.0-sample',
// and there is no honest ordering between a placeholder and a real build.
if (!reportedparts || !minimumparts) return false
for (let i = 0; i < 4; i += 1) {
if (reportedparts[i] > minimumparts[i]) return true
if (reportedparts[i] < minimumparts[i]) return false
}
return true
}
const agentSupportsLogCollection = computed(() => {
// No floor served means the action carries no version requirement, so this
// must not block. Only a floor that the endpoint fails should disable it.
if (!collectLogMinAgent.value) return true
return agentMeetsMinimum(agentversion.value, collectLogMinAgent.value)
})
// Names both versions. "Requires a newer agent" alone leaves the operator
// without the one fact they need to act: which version this machine is on.
const logCollectorVersionNotice = computed(() => {
if (agentSupportsLogCollection.value) return ''
const reported = agentversion.value
? `this endpoint reports ${agentversion.value}`
: 'this endpoint has not reported an agent version'
return `Log collection requires agent ${collectLogMinAgent.value} or newer; `
+ `${reported}. Update the agent on this endpoint to use it.`
})
// History for this endpoint. Metadata only - bodies are fetched one at a time,
// because 20 rows of multi-hundred-KB logs would make opening the tab far more
// expensive than reading the one log that is actually wanted.
@@ -1838,6 +1921,12 @@ function validateLogPath(raw) {
}
async function collectLog() {
// Belt and braces with the disabled button: keyboard Enter reaches here too,
// and a stale agentversion could leave the button live for a moment.
if (!agentSupportsLogCollection.value) {
logpatherror.value = logCollectorVersionNotice.value
return
}
const path = logpath.value.trim()
logpatherror.value = validateLogPath(path)
if (logpatherror.value) return

View File

@@ -0,0 +1,109 @@
"""Register staged endpoint agent builds as installable Applications.
The endpoint update_app push resolves an install source from the applications
table alone - it has no knowledge of the agent version store. Without a row
there, a build can be staged and current in agentstore and still be impossible
to push, which is exactly the gap found in the fleet: no application row existed
for the agent at all.
This module bridges the two by making the UPLOAD the single source of truth.
Staging a build repackages the current agent (plus tray) and rewrites the
application row's installpath and installersha256 in the same request, so the
row cannot describe a build that is no longer current. Nothing here is
hand-maintained; an operator never edits these rows.
"""
import os
from shopdb.api import Application, application_package_location, db
from . import agentstore
# Reserved appname for the managed agent bundle. appname is unique, which is
# what makes the upsert idempotent: repeated uploads update one row rather than
# accumulating a row per build. Changing this string strands the existing row,
# so it is a constant rather than anything configurable.
AGENT_APPNAME = 'ShopDB Endpoint Agent'
AGENT_PACKAGE_FILENAME = 'shopdb-endpoint-agent.zip'
_AGENT_DESCRIPTION = (
'Managed ShopDB endpoint agent bundle. Created and kept current '
'automatically when an agent build is staged - do not edit by hand.'
)
def _package_path(appid):
directory, _prefix = application_package_location()
return os.path.join(directory, str(appid), AGENT_PACKAGE_FILENAME)
def _get_or_create_app():
"""Return the reserved agent Application row, creating it when absent.
Does not commit: the caller commits once, after the package is written, so
a failed write cannot leave a row advertising an installer that is not
there.
"""
app = Application.query.filter_by(appname=AGENT_APPNAME).first()
if app:
return app, False
app = Application(
appname=AGENT_APPNAME,
appdescription=_AGENT_DESCRIPTION,
# Installable so it can be pushed; hidden so a machine-managed row does
# not surface in the software catalogue alongside real licensed apps.
isinstallable=True,
ishidden=True,
)
db.session.add(app)
# Flush rather than commit: the row needs its appid to build the package
# path, but the transaction stays open so the whole registration is atomic.
db.session.flush()
return app, True
def register_current_agent_package():
"""Repackage the current agent build and point the agent app row at it.
Returns a dict describing what was registered, or None when there is no
current agent build to package. Safe to call on every upload: it is an
upsert keyed on the reserved appname, and it rewrites the package file in
place.
"""
payload, manifest = agentstore.build_current_package()
if payload is None:
return None
app, created = _get_or_create_app()
path = _package_path(app.appid)
os.makedirs(os.path.dirname(path), exist_ok=True)
# Write to a temp name and replace, so an interrupted write cannot leave a
# truncated archive at the path the row already advertises. os.replace is
# atomic within a filesystem.
temppath = path + '.tmp'
with open(temppath, 'wb') as handle:
handle.write(payload)
os.replace(temppath, path)
_directory, urlprefix = application_package_location()
app.installpath = '{}{}/{}'.format(
urlprefix, app.appid, AGENT_PACKAGE_FILENAME)
# The digest of the ARCHIVE, which is what the agent downloads and verifies.
# The per-binary digests inside the manifest are for the install script to
# check its unpacked contents, and are not interchangeable with this one.
app.installersha256 = manifest['sha256']
db.session.commit()
return {
'appid': app.appid,
'appname': app.appname,
'created': created,
'installpath': app.installpath,
'sha256': manifest['sha256'],
'size': manifest['size'],
'agentversion': manifest['agentversion'],
'contents': manifest['contents'],
}

View File

@@ -35,6 +35,7 @@ import json
import os
import re
import struct
import zipfile
from datetime import datetime
from flask import current_app
@@ -470,3 +471,109 @@ def set_current(binkey, version):
manifest['current'] = version
_write_manifest(binkey, manifest)
return True
def version_sha256(binkey, version):
"""The recorded digest for a staged version, or None when not recorded.
Read from the manifest rather than recomputed: store_version already
hashed the exact bytes it wrote, so re-reading the file to hash it again
would only introduce a way for the two to disagree.
"""
for entry in _read_manifest(binkey).get('versions', []):
if entry.get('version') == version:
return entry.get('sha256')
return None
# Name the upgrade payload carries inside the archive. Fixed rather than derived
# from the staged filenames so the install script can address its contents
# without being told what to expect.
PACKAGE_MEMBER_NAMES = {
'agent': 'shopdbagent.exe',
'tray': 'shopdb-endpoint-tray.exe',
}
# Fixed member timestamp so packaging the same build twice yields byte-identical
# archives. The value is arbitrary (the DOS epoch zip supports); only its
# constancy matters. Real build provenance lives in the manifest's versions and
# digests, not in a zip entry's mtime.
_PACKAGE_TIMESTAMP = (1980, 1, 1, 0, 0, 0)
def build_current_package():
"""Zip the current agent build (plus the current tray) into one archive.
Returns (bytes, manifestdict) or (None, None) when no agent build is
current. The agent is required; the tray is included only when it too has
a current staged version, because an endpoint can legitimately run the
service with no tray and half a tray upgrade is worse than none.
The archive is built IN MEMORY and returned rather than written here: the
caller decides where it lands, and nothing under the version store gets a
derived artifact that retention would then have to reason about.
The returned manifest describes what went in - version and digest per
member - so the install script can verify it unpacked the build the server
meant to send, rather than trusting the archive's filename.
"""
agentversion = current_version('agent')
if not agentversion:
return None, None
members = []
agentpath = _version_exe_path('agent', agentversion)
if not os.path.isfile(agentpath):
return None, None
members.append(('agent', agentversion, agentpath))
trayversion = current_version('tray')
if trayversion:
traypath = _version_exe_path('tray', trayversion)
if os.path.isfile(traypath):
members.append(('tray', trayversion, traypath))
contents = {}
buffer = io.BytesIO()
# ZIP_DEFLATED, not ZIP_STORED: these are ~70 MB self-contained binaries
# carrying the whole .NET runtime, and the archive crosses the wire to
# every endpoint being upgraded.
with zipfile.ZipFile(buffer, 'w', compression=zipfile.ZIP_DEFLATED) as archive:
for binkey, version, path in members:
try:
with open(path, 'rb') as handle:
data = handle.read()
except OSError:
# A staged file that cannot be read means the package would be
# silently incomplete. Refuse the whole thing: a partial upgrade
# payload is how an endpoint ends up with a new service exe and
# a stale tray.
return None, None
name = PACKAGE_MEMBER_NAMES[binkey]
# Build the entry with a FIXED timestamp. zipfile otherwise stamps
# each member with the current local time, which makes the archive
# bytes - and therefore its digest - different on every call for
# identical input. update_app pins a download to a digest, so a
# drifting hash would invalidate a package an endpoint is already
# fetching. Same reason the mode is set explicitly.
info = zipfile.ZipInfo(name, date_time=_PACKAGE_TIMESTAMP)
info.compress_type = zipfile.ZIP_DEFLATED
info.external_attr = 0o644 << 16
archive.writestr(info, data)
contents[binkey] = {
'filename': name,
'version': version,
# Prefer the recorded digest so the package agrees with the
# manifest; fall back to hashing when an older manifest entry
# predates digest recording.
'sha256': (version_sha256(binkey, version)
or hashlib.sha256(data).hexdigest()),
}
payload = buffer.getvalue()
return payload, {
'agentversion': agentversion,
'contents': contents,
'sha256': hashlib.sha256(payload).hexdigest(),
'size': len(payload),
}

View File

@@ -15,6 +15,12 @@ Authorization boundary (deliberate):
command family from application deploys (endpoints.deploy) so app-push rights
never imply SYSTEM-level agent replacement. That push pipe is a later slice.
Staging also keeps the pushable application row current (see agentregistry):
whenever the current build changes, the bundle is repackaged and the row's
installpath/installersha256 are rewritten. Registration is deliberately NOT
allowed to fail the upload - the binary is already staged and audited by then,
so a packaging problem is reported in the response payload rather than raised.
Uploads are capped and PE-checked so a wrong or hostile file cannot be staged.
"""
@@ -24,7 +30,7 @@ from shopdb.api import (success_response, error_response, ErrorCodes, AuditLog,
require_permission)
from .routes import endpoints_bp
from . import agentstore
from . import agentstore, agentregistry
# Hard cap on an uploaded agent binary. This is a generous ceiling that still
# refuses an operator who points at a giant wrong file before it is buffered
@@ -40,6 +46,33 @@ from . import agentstore
_MAX_UPLOAD_BYTES = 128 * 1024 * 1024
def _sync_agent_registration():
"""Repackage and re-register the current build. Returns a status dict.
Called from every route that can change which build is current, so the
pushable application row can never describe a build that is no longer
staged. That drift is the whole failure this is meant to prevent, and it
would be silent - the row would keep pointing at bytes nobody selected.
Never raises. Staging has already succeeded on disk and been audited by the
time this runs; a packaging or database problem must not turn a successful
upload into an error response and leave the operator thinking the binary
was rejected. The failure is reported in the response payload instead, so
the UI can surface it without losing the upload.
"""
try:
registration = agentregistry.register_current_agent_package()
except Exception as exc: # noqa: BLE001 - reported, not swallowed
db.session.rollback()
return {'ok': False, 'error': str(exc)}
if registration is None:
# No current agent build to package. Normal when only a tray has been
# staged: the bundle is defined by the agent, so there is nothing to
# register yet.
return {'ok': False, 'error': 'no current agent build to package'}
return {'ok': True, **registration}
@endpoints_bp.route('/siteagent/binaries/<binkey>', methods=['GET'])
@require_permission('endpoints.agentpackage')
def list_agent_binaries(binkey):
@@ -129,10 +162,16 @@ def upload_agent_binary(binkey):
'madecurrent': makecurrent})
db.session.commit()
# Keep the pushable application row in step with what is now current, in
# the same request. Doing it here rather than at push time means the row
# and the staged build cannot disagree in between.
registration = _sync_agent_registration()
return success_response({
'binary': binkey,
'current': result['current'],
'versions': result['versions'],
'registration': registration,
})
@@ -215,10 +254,12 @@ def adopt_legacy_agent_binary(binkey):
ErrorCodes.VALIDATION_ERROR,
'the legacy binary could not be adopted',
http_code=400)
registration = _sync_agent_registration()
return success_response({
'binary': binkey,
'current': result['current'],
'versions': result['versions'],
'registration': registration,
})
@@ -243,8 +284,13 @@ def set_current_agent_binary(binkey):
ErrorCodes.NOT_FOUND,
'that version is not staged',
http_code=404)
# A rollback changes the bytes the fleet should receive; repackage so the
# application row follows the pointer instead of pinning the build the
# operator just rolled away from.
registration = _sync_agent_registration()
return success_response({
'binary': binkey,
'current': agentstore.current_version(binkey),
'versions': agentstore.list_versions(binkey),
'registration': registration,
})

View File

@@ -31,7 +31,8 @@ from ..models import (
EndpointState, EndpointEvent, EndpointCommand, EndpointApplication,
EndpointHealthState, OperatorReport, EndpointLogCollection,
)
from ..commands import COMMAND_ALLOWLIST, is_allowed_action, ALLOWED_ACTIONS_FOR, row_actions_for, requires_target, action_targetkind, elevated_permission_for, PLACE_POLICY_HOLD, ACTIVE_POLICY_HOLDS, HOLDABLE_ACTIONS_FOR, RELEASE_POLICY_HOLD, ACTIVE_LOCK_FOR, PLACE_ENDPOINT_LOCK, RELEASE_ENDPOINT_LOCK
from ..commands import COMMAND_ALLOWLIST, is_allowed_action, ALLOWED_ACTIONS_FOR, row_actions_for, requires_target, action_targetkind, elevated_permission_for, minimum_agent_version_for, action_minimum_versions, parse_agent_version, agent_meets_minimum, PLACE_POLICY_HOLD, ACTIVE_POLICY_HOLDS, HOLDABLE_ACTIONS_FOR, RELEASE_POLICY_HOLD, ACTIVE_LOCK_FOR, PLACE_ENDPOINT_LOCK, RELEASE_ENDPOINT_LOCK
from . import agentstore
from ..eventcatalog import CRITICAL_EVENT_IDS, event_severity, event_title
from shopdb.api import send_alert
from markupsafe import escape
@@ -432,6 +433,14 @@ def get_endpoint(asset_id: int):
data = state.to_dict()
data['status'] = status
data['uptimedisplay'] = _format_uptime(state.uptimeseconds)
# The currently published agent build, so the page can say "behind" without
# hardcoding a version that goes stale the next time one is published.
# None when nothing has been staged yet, which reads as "cannot tell".
data['agentcurrent'] = agentstore.current_version('agent')
# Per-action agent floors. Served rather than duplicated in the browser:
# the enqueue guard refuses against these same numbers, so a copy in the
# frontend would silently disagree the moment a floor was raised here.
data['actionminimums'] = action_minimum_versions()
return success_response(data)
@@ -553,6 +562,19 @@ def panel_status(asset_id: int):
entry['format'] = fmt
return entry
# Annotate the agent version when the endpoint is behind the published
# build. Done here rather than as a separate row so the fact sits on the
# version it describes. Silent when nothing is published, when the endpoint
# has not reported, or when either version is unparseable - claiming
# "behind" from a placeholder like '0.0.0-sample' would be a false alarm.
agentvalue = state.agentversion
published = agentstore.current_version('agent')
if (agentvalue and published
and parse_agent_version(agentvalue)
and parse_agent_version(published)
and not agent_meets_minimum(agentvalue, published)):
agentvalue = '{} (behind {})'.format(agentvalue, published)
fields = [
field('Logged in user', state.loggedinuser),
field('Uptime', _format_uptime(state.uptimeseconds)),
@@ -562,7 +584,7 @@ def panel_status(asset_id: int):
fmt='datetime'),
field('CPU %', state.cpupercent),
field('Memory %', state.memorypercent),
field('Agent', state.agentversion, mono=True),
field('Agent', agentvalue, mono=True),
field('Tray', state.trayversion, mono=True),
field('Last reported',
state.lastreportedat.isoformat() + 'Z' if state.lastreportedat else None,
@@ -2852,6 +2874,29 @@ def enqueue_endpoint_command(asset_id: int):
ErrorCodes.VALIDATION_ERROR,
f'Action {action!r} does not accept a target')
# Capability, not authorization: does the agent on THIS endpoint actually
# implement the action? An agent older than the action falls through to its
# unknown-action default and answers "Unknown or disallowed action", which
# reads as a permission refusal and sends the operator to check a grant they
# already hold. Checked after the permission and policy gates so an
# unauthorized caller still gets the authorization answer rather than a
# version hint about a machine they may not act on.
#
# Deliberately a hard refusal, not a warning: the command is guaranteed to
# fail, and queueing it anyway spends the one in-flight command slot and
# writes a failed row that looks like the endpoint misbehaved.
minagentversion = minimum_agent_version_for(action)
if minagentversion:
state = EndpointState.query.filter_by(assetid=asset_id).first()
reportedversion = getattr(state, 'agentversion', None)
if not agent_meets_minimum(reportedversion, minagentversion):
return error_response(
ErrorCodes.VALIDATION_ERROR,
f'Action {action!r} requires agent {minagentversion} or newer; '
f'this endpoint reports '
f'{reportedversion or "no version"}. Update the agent on this '
f'endpoint to use it.')
# update_app pins a specific application version; verify it resolves to a
# real installer now so a broken push is rejected at enqueue rather than
# failing opaquely on the endpoint minutes later.

View File

@@ -127,6 +127,14 @@ COMMAND_ALLOWLIST = {
'still reads a caller-supplied path as SYSTEM: it is '
'bounded to text files, but it is not a sandbox.',
'targetkind': 'logpath',
# Introduced in agent 1.16.0. An older agent has no case for this action
# and falls through to its unknown-action default, which answers
# "Unknown or disallowed action" - a message that reads as an
# authorization refusal and sends the operator looking at permissions
# they already hold. Declaring the floor here lets the enqueue route
# refuse first, naming both versions, so the answer is the actionable
# one: this endpoint needs a newer agent.
'minagentversion': '1.16.0',
},
'update_app': {
'label': 'Install/update application',
@@ -230,6 +238,70 @@ def elevated_permission_for(action):
return ELEVATED_ACTION_PERMISSIONS.get(action)
def minimum_agent_version_for(action):
"""The lowest agent version that implements an action, or None if any will do."""
return COMMAND_ALLOWLIST.get(action, {}).get('minagentversion')
def action_minimum_versions():
"""Every action that has an agent version floor, as {action: version}.
Actions without a floor are omitted rather than mapped to None: a caller
asking "is there a minimum for this?" gets a plain absent key, and the
payload stays small as the allowlist grows.
"""
return {
action: spec['minagentversion']
for action, spec in COMMAND_ALLOWLIST.items()
if spec.get('minagentversion')
}
def parse_agent_version(value):
"""A reported agent version as a comparable tuple, or None if unusable.
Agents report four parts ('1.15.0.0') while the csproj and the binary store
carry three ('1.16.0'), so a plain string compare puts a NEWER agent below
the floor it satisfies. Both are normalised to a tuple and compared
numerically, with missing trailing parts treated as zero, so '1.16.0' and
'1.16.0.0' are the same version.
Anything non-numeric returns None rather than a guess. The fleet carries
'0.0.0-sample' on seeded rows, and there is no honest ordering between a
placeholder and a real build - the caller decides what to do about an
unknown version, which is a different question from being out of date.
"""
if not value:
return None
parts = []
for chunk in str(value).strip().split('.'):
if not chunk.isdigit():
return None
parts.append(int(chunk))
if not parts:
return None
while len(parts) < 4:
parts.append(0)
return tuple(parts[:4])
def agent_meets_minimum(reported, minimum):
"""Whether a reported agent version satisfies a minimum.
Returns False when either side is unparseable. That is the fail-closed
direction: an endpoint whose version we cannot read is one we cannot claim
supports the action, and letting it through only moves the failure to the
agent where the error message is worse.
"""
if not minimum:
return True
reportedparts = parse_agent_version(reported)
minimumparts = parse_agent_version(minimum)
if reportedparts is None or minimumparts is None:
return False
return reportedparts >= minimumparts
# Maps a monitored services/processes row to the start/stop/status actions that
# control it, so the services panel can render an inline toggle on exactly the
# rows that have a registered command action. The key is the agent-reported

View File

@@ -61,6 +61,16 @@
</select>
</div>
<!-- Placeholder for the staged agent rollout. Shown only to operators who
hold the grant it will require, so it reads as "not built yet" to the
people waiting on it rather than as a missing permission to everyone
else. This page is due a refactor; the utility lands with that work. -->
<div v-if="canagentupdate" class="rollout-note">
Bulk Agent Update utility coming soon. Until then, agents update through
the existing installer push, and an endpoint behind the published build is
flagged on its own detail page.
</div>
<!-- Actions live here rather than per row. A tenth column of buttons
pushed itself off the edge of a nowrap table, and selecting one row
costs a single click while giving one command path instead of two. -->
@@ -194,6 +204,9 @@ const toast = useToast()
// Gated in the UI for convenience only; every enqueue is re-checked server-side.
const cancommand = computed(() => authStore.hasPermission('endpoints.command'))
const canrestart = computed(() => authStore.hasPermission('endpoints.restart'))
// Reserved for the staged rollout. Agent replacement is the control plane
// modifying itself, so it stays separate from ordinary app deployment.
const canagentupdate = computed(() => authStore.hasPermission('endpoints.agentupdate'))
const allselected = computed(() =>
endpoints.value.length > 0 && selected.value.length === endpoints.value.length)
@@ -515,6 +528,18 @@ function changePerPage(newPerPage) {
.bulk-count { font-size: 0.9rem; margin-right: 0.25rem; }
/* Deliberately quieter than .bulk-bar: this announces something that does not
exist yet, so it must not compete with controls that actually do work. */
.rollout-note {
padding: 0.6rem 0.9rem;
margin-bottom: 0.75rem;
font-size: 0.9rem;
color: var(--text-light);
background: var(--bg-card);
border: 1px dashed var(--border);
border-radius: 0.25rem;
}
.select-cell { width: 1%; }

View File

@@ -99,6 +99,25 @@ def collector_authorized(pluginname: str):
from shopdb.core.api.collector import _check_collector_auth, _plugin_api_key
return _check_collector_auth(_plugin_api_key(pluginname), pluginname)
def application_package_location():
"""Where application installers live, and the URL prefix that serves them.
Returns (directory, urlprefix). A plugin that registers an installable
payload (the endpoint agent bundle, for example) must put the file exactly
where the applications API expects it and build installpath the same way,
or update_app cannot resolve the download. Reconstructing either by hand
lets the two drift silently, so both come from one place.
Imported lazily for the same reason as collector_authorized above:
shopdb.core.api imports back into shopdb.api, so a module-level import
would be circular at load time.
"""
from shopdb.core.api.applications import (
_apppackage_dir, APP_PACKAGE_URL_PREFIX)
return _apppackage_dir(), APP_PACKAGE_URL_PREFIX
# Serving an uploaded file safely. A plugin that serves user-supplied bytes must
# not have to remember the headers that keep an SVG or an HTML file from running
# as script in this origin - see shopdb/utils/uploads.py.
@@ -388,6 +407,7 @@ __all__ = [
'list_service_tokens',
'revoke_service_tokens',
'collector_authorized',
'application_package_location',
'SupportTeam',
# Serving uploads
'send_upload',

View File

@@ -0,0 +1,250 @@
"""Auto-registration of staged agent builds as a pushable Application.
The endpoint update_app push resolves an install source from the applications
table alone; it cannot see the agent version store. Before this, a build could
be staged and current while no application row existed at all, which is exactly
what was found in the fleet - the agent was unpushable.
These tests pin the properties that make the bridge trustworthy:
- staging creates the row, and re-staging updates it in place (never a row
per build, because appname is unique and drift is the failure mode);
- the packaged archive is byte-stable for unchanged input, because update_app
pins a download to a digest and a drifting hash breaks verification;
- every path that changes 'current' re-registers, including rollback;
- registration failure never fails an upload that already landed on disk.
Uploads use PE-magic bytes with a monkeypatched version parser, matching the
approach in test_endpoints_agentupload.py - building a real PE with an
RT_VERSION resource inline is impractical.
Every test runs against an ISOLATED store. create_app('testing') resolves
instance_path to the real repo instance/ directory, so without redirection
these tests stage fake 130-byte binaries into the developer's actual agent
store and leave invented versions behind in its manifest. The isolate fixture
below is autouse for that reason: it is a correctness requirement, not a
convenience.
"""
import hashlib
import io
import os
import zipfile
import pytest
from plugins.endpoints.api import agentstore, agentupload, agentregistry
from shopdb.core.models.application import Application
UPLOAD_URL = '/api/endpoints/siteagent/binaries/{}'
CURRENT_URL = '/api/endpoints/siteagent/binaries/{}/current'
_PE_BYTES = b'MZ' + b'\x00' * 128
@pytest.fixture(autouse=True)
def isolate_store(tmp_path, monkeypatch):
"""Point the agent store and package output at a per-test directory.
Both must be redirected. Redirecting only the store would still write the
~zip into the real applicationpackages tree under a live application id.
"""
siteagent = tmp_path / 'siteagent'
packages = tmp_path / 'applicationpackages'
siteagent.mkdir()
packages.mkdir()
monkeypatch.setattr(agentstore, '_siteagent_dir', lambda: str(siteagent))
monkeypatch.setattr(agentregistry, 'application_package_location',
lambda: (str(packages), '/api/applications/package/'))
return tmp_path
def _upload(client, headers, monkeypatch, version, binkey='agent',
payload=None, makecurrent=None):
"""Stage a build through the real HTTP route."""
monkeypatch.setattr(agentstore, 'read_pe_file_version', lambda data: version)
form = {'file': (io.BytesIO(payload or _PE_BYTES), 'shopdbagent.exe'),
'version': version}
if makecurrent is not None:
form['makecurrent'] = makecurrent
return client.post(UPLOAD_URL.format(binkey), data=form,
content_type='multipart/form-data', headers=headers)
def _agent_row():
return Application.query.filter_by(
appname=agentregistry.AGENT_APPNAME).first()
def test_staging_a_build_creates_a_pushable_application_row(
client, db, app, auth_headers, monkeypatch):
"""Without this row, update_app has nothing to resolve and cannot push."""
assert _agent_row() is None
response = _upload(client, auth_headers, monkeypatch, '1.16.0.0')
assert response.status_code == 200
registration = response.get_json()['data']['registration']
assert registration['ok'] is True
row = _agent_row()
assert row is not None
# The row must be resolvable and pinned: a path update_app can fetch, and a
# digest it can verify the bytes against.
assert row.installpath == '/api/applications/package/{}/{}'.format(
row.appid, agentregistry.AGENT_PACKAGE_FILENAME)
assert row.installersha256 == registration['sha256']
assert row.isinstallable is True
# Hidden: this is machine-managed, and must not appear in the software
# catalogue beside real licensed applications.
assert row.ishidden is True
def test_restaging_updates_the_same_row_rather_than_adding_one(
client, db, app, auth_headers, monkeypatch):
"""appname is unique; a row per build would break the push target."""
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
first = _agent_row()
firstid = first.appid
response = _upload(client, auth_headers, monkeypatch, '1.17.0.0')
assert response.status_code == 200
assert Application.query.filter_by(
appname=agentregistry.AGENT_APPNAME).count() == 1
row = _agent_row()
assert row.appid == firstid
# The row must follow the newly current build, not keep pointing at 1.16.
assert row.installersha256 == \
response.get_json()['data']['registration']['sha256']
def test_packaging_the_same_build_twice_yields_the_same_digest(
client, db, app, auth_headers, monkeypatch):
"""update_app pins bytes by digest.
zipfile stamps members with the current mtime by default, which made the
archive hash change on every call for identical input. That would rewrite
installersha256 out from under an endpoint mid-download and fail its
SHA-256 verification.
"""
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
firstpayload, firstmanifest = agentstore.build_current_package()
secondpayload, secondmanifest = agentstore.build_current_package()
assert firstmanifest['sha256'] == secondmanifest['sha256']
assert firstpayload == secondpayload
def test_the_package_contains_the_staged_binary_under_a_fixed_name(
client, db, app, auth_headers, monkeypatch):
"""The install script addresses members by name, not by discovery."""
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
payload, manifest = agentstore.build_current_package()
with zipfile.ZipFile(io.BytesIO(payload)) as archive:
assert archive.testzip() is None
names = archive.namelist()
assert agentstore.PACKAGE_MEMBER_NAMES['agent'] in names
assert manifest['agentversion'] == '1.16.0.0'
assert manifest['contents']['agent']['version'] == '1.16.0.0'
def test_the_written_package_file_matches_the_recorded_digest(
client, db, app, auth_headers, monkeypatch):
"""A row whose digest disagrees with the file on disk is unpushable."""
_upload(client, auth_headers, monkeypatch, '1.16.0.0')
row = _agent_row()
path = agentregistry._package_path(row.appid)
assert os.path.isfile(path)
with open(path, 'rb') as handle:
ondisk = hashlib.sha256(handle.read()).hexdigest()
assert ondisk == row.installersha256
# The atomic-write temp file must not survive a successful write.
assert not os.path.isfile(path + '.tmp')
def test_rolling_back_repoints_the_row_at_the_restored_build(
client, db, app, auth_headers, monkeypatch):
"""A rollback changes the bytes the fleet should get.
If only upload re-registered, the row would still pin the build the
operator just rolled away from, and the push would ship it anyway.
"""
_upload(client, auth_headers, monkeypatch, '1.16.0.0',
payload=b'MZ' + b'\x01' * 256)
oldsha = _agent_row().installersha256
_upload(client, auth_headers, monkeypatch, '1.17.0.0',
payload=b'MZ' + b'\x02' * 256)
newsha = _agent_row().installersha256
assert newsha != oldsha
response = client.put(CURRENT_URL.format('agent'),
json={'version': '1.16.0.0'}, headers=auth_headers)
assert response.status_code == 200
assert response.get_json()['data']['registration']['ok'] is True
assert agentstore.current_version('agent') == '1.16.0.0'
# Back to the rolled-to build's package, not the newer one.
assert _agent_row().installersha256 == oldsha
def test_a_registration_failure_does_not_fail_the_upload(
client, db, app, auth_headers, monkeypatch):
"""The binary is already staged and audited by the time this runs.
Turning a packaging problem into an error response would tell the operator
the upload was rejected when it actually succeeded, inviting a re-upload of
a build that is already there.
"""
def _explode():
raise OSError('simulated packaging failure')
monkeypatch.setattr(agentregistry.agentstore, 'build_current_package',
_explode)
response = _upload(client, auth_headers, monkeypatch, '1.16.0.0')
assert response.status_code == 200
body = response.get_json()['data']
# The staging itself still stands...
assert body['current'] == '1.16.0.0'
# ...and the failure is surfaced rather than swallowed.
assert body['registration']['ok'] is False
assert 'simulated packaging failure' in body['registration']['error']
def test_staging_only_a_tray_reports_that_there_is_nothing_to_package(
client, db, app, auth_headers, monkeypatch):
"""The bundle is defined by the agent; a tray alone is not pushable."""
response = _upload(client, auth_headers, monkeypatch, '1.6.5.0',
binkey='tray')
assert response.status_code == 200
registration = response.get_json()['data']['registration']
assert registration['ok'] is False
assert 'no current agent build' in registration['error']
assert _agent_row() is None
def test_these_tests_do_not_touch_the_real_agent_store(
client, db, app, auth_headers, monkeypatch, isolate_store):
"""Guard the isolation itself.
An earlier draft of this file wrote fake 130-byte binaries into the repo's
real instance/siteagent/ tree and invented a version in its manifest,
because create_app('testing') shares the developer's instance_path. If that
redirection ever regresses, fail here rather than silently corrupting a
staged build again.
"""
_upload(client, auth_headers, monkeypatch, '9.9.9.9')
# The staged build landed inside the per-test directory...
staged = isolate_store / 'siteagent' / 'agent' / '9.9.9.9'
assert staged.is_dir()
# ...and the real store never saw it.
realstore = os.path.join(app.instance_path, 'siteagent', 'agent', '9.9.9.9')
assert not os.path.exists(realstore)

View File

@@ -0,0 +1,158 @@
"""The agent version guard on command enqueue.
An agent older than an action has no case for it and falls through to its
unknown-action default, answering "Unknown or disallowed action". That reads as
an authorization refusal, so the operator goes looking at permissions they
already hold - which is exactly what happened on a live endpoint running 1.15.0
after collect_log shipped in 1.16.0.
These tests pin the refusal at the SERVER, before the command is ever queued,
and pin that the message names both versions so the answer is actionable.
"""
from werkzeug.security import generate_password_hash
def _endpoint(db, hostname, agentversion):
"""An Asset + Computer pair whose reported state carries an agent version."""
from shopdb.api import Asset, AssetType
from plugins.computers.models import Computer
from plugins.endpoints.models import EndpointState
assettype = AssetType.query.filter_by(assettype='computer').first()
if assettype is None:
assettype = AssetType(assettype='computer', pluginname='computers',
tablename='computers')
db.session.add(assettype)
db.session.flush()
asset = Asset(assetnumber=hostname, name=hostname,
assettypeid=assettype.assettypeid, isactive=True)
db.session.add(asset)
db.session.flush()
db.session.add(Computer(assetid=asset.assetid, hostname=hostname))
db.session.add(EndpointState(assetid=asset.assetid, hostname=hostname,
agentversion=agentversion))
db.session.commit()
return asset.assetid
def _admin(db, client, username='guardadmin'):
"""An admin, so permission and policy gates cannot be what refuses."""
from shopdb.core.models import User, Role, Permission
Permission.seed()
db.session.flush()
role = Role.query.filter_by(rolename='admin').first()
if role is None:
role = Role(rolename='admin')
db.session.add(role)
db.session.commit()
user = User(username=username, email=username + '@test.local',
passwordhash=generate_password_hash('testpass'))
user.roles.append(role)
db.session.add(user)
db.session.commit()
response = client.post('/api/auth/login',
json={'username': username, 'password': 'testpass'})
token = response.get_json()['data']['access_token']
return {'Authorization': f'Bearer {token}'}
def _collect(client, headers, assetid):
return client.post(f'/api/endpoints/{assetid}/commands',
json={'action': 'collect_log',
'target': r'C:\logs\app.log'},
headers=headers)
def test_an_agent_below_the_minimum_is_refused_at_enqueue(app, client, db):
"""The command must not be queued at all.
Queueing it would spend the endpoint's one in-flight command slot and leave
a failed row that reads as endpoint misbehaviour, when the truth is the
agent simply predates the feature.
"""
from plugins.endpoints.models import EndpointCommand
assetid = _endpoint(db, 'oldagentbox', '1.15.0.0')
headers = _admin(db, client)
response = _collect(client, headers, assetid)
assert response.status_code == 400
message = response.get_json()['data']['error']['message']
# Both versions, or the operator cannot tell what to do about it.
assert '1.16.0' in message
assert '1.15.0.0' in message
assert EndpointCommand.query.filter_by(assetid=assetid).count() == 0
def test_an_agent_at_the_minimum_is_allowed(app, client, db):
"""The guard must not refuse a capable endpoint.
1.16.0.0 against a 1.16.0 floor is the four-vs-three-part spelling of the
same build; a lexical compare would get this wrong.
"""
from plugins.endpoints.models import EndpointCommand
assetid = _endpoint(db, 'newagentbox', '1.16.0.0')
headers = _admin(db, client)
response = _collect(client, headers, assetid)
assert response.status_code in (200, 201)
assert EndpointCommand.query.filter_by(assetid=assetid,
action='collect_log').count() == 1
def test_a_lexically_larger_but_older_version_is_still_refused(app, client, db):
"""'1.9.0.0' must not pass a '1.16.0' floor.
String comparison says '9' > '1' and lets it through. This is the specific
bug the numeric parse exists to prevent, so it gets its own test rather
than living only in the helper's unit coverage.
"""
assetid = _endpoint(db, 'lexicaltrapbox', '1.9.0.0')
headers = _admin(db, client)
response = _collect(client, headers, assetid)
assert response.status_code == 400
assert '1.16.0' in response.get_json()['data']['error']['message']
def test_an_unknown_agent_version_is_refused_rather_than_guessed(app, client, db):
"""A placeholder version fails closed.
Seeded rows carry '0.0.0-sample'. There is no honest ordering between a
placeholder and a real build, and letting it through only moves the failure
to the agent where the error message is the misleading one.
"""
assetid = _endpoint(db, 'samplebox', '0.0.0-sample')
headers = _admin(db, client)
response = _collect(client, headers, assetid)
assert response.status_code == 400
def test_an_action_without_a_floor_is_unaffected(app, client, db):
"""The guard is per-action, not global.
collect_now declares no minimum, so an old agent must still be able to run
it - otherwise adding a floor to one action quietly narrows the others.
"""
from plugins.endpoints.models import EndpointCommand
assetid = _endpoint(db, 'refreshbox', '1.11.0.0')
headers = _admin(db, client)
response = client.post(f'/api/endpoints/{assetid}/commands',
json={'action': 'collect_now'}, headers=headers)
assert response.status_code in (200, 201)
assert EndpointCommand.query.filter_by(assetid=assetid,
action='collect_now').count() == 1

View File

@@ -11,7 +11,8 @@ from plugins.endpoints.plugin import EndpointsPlugin, CURATED_EVENT_IDS
from plugins.endpoints.eventcatalog import EVENT_CATALOG, CRITICAL_EVENT_IDS
from plugins.endpoints.commands import (
COMMAND_ALLOWLIST, is_allowed_action, elevated_permission_for,
requires_target,
requires_target, minimum_agent_version_for, agent_meets_minimum,
parse_agent_version,
)
@@ -216,6 +217,61 @@ def test_collect_log_survives_a_policy_that_never_mentions_it():
assert elevated_permission_for('collect_log') == 'endpoints.updateapp'
def test_collect_log_declares_the_agent_version_that_implements_it():
"""collect_log names its minimum agent version.
An agent older than the action has no case for it and falls through to its
unknown-action default, answering "Unknown or disallowed action" - which
reads as an authorization refusal and sends the operator to check a grant
they already hold. The floor is what lets the server refuse first with the
actionable answer instead.
"""
assert minimum_agent_version_for('collect_log') == '1.16.0'
# Actions that every shipped agent understands must NOT declare a floor,
# or the guard starts refusing endpoints that are perfectly capable.
assert minimum_agent_version_for('collect_now') is None
assert minimum_agent_version_for('system_restart') is None
def test_agent_version_comparison_survives_real_fleet_version_strings():
"""Version comparison is numeric per part, not lexical.
Two traps, both real in this fleet. Agents report four parts ('1.15.0.0')
while the csproj and binary store carry three ('1.16.0'), so the same build
must compare equal across both spellings. And a lexical compare ranks
'1.9.0.0' ABOVE '1.16.0' because '9' > '1' - which would silently let an
old agent past the guard, the exact failure the guard exists to prevent.
"""
assert parse_agent_version('1.16.0') == parse_agent_version('1.16.0.0')
assert agent_meets_minimum('1.16.0', '1.16.0')
assert agent_meets_minimum('1.16.0.0', '1.16.0')
assert agent_meets_minimum('1.17.0.0', '1.16.0')
assert agent_meets_minimum('2.0.0.0', '1.16.0')
assert not agent_meets_minimum('1.15.0.0', '1.16.0')
assert not agent_meets_minimum('1.11.0.0', '1.16.0')
# The lexical trap.
assert not agent_meets_minimum('1.9.0.0', '1.16.0')
def test_an_unreadable_agent_version_fails_the_guard_closed():
"""An unparseable or absent version does not satisfy a minimum.
Eight endpoints in this fleet report '0.0.0-sample', and a never-checked-in
endpoint reports nothing at all. There is no honest ordering between a
placeholder and a real build, so the guard refuses rather than guessing.
Letting it through would only move the failure to the agent, where the
error message is the misleading one.
"""
assert parse_agent_version('0.0.0-sample') is None
assert not agent_meets_minimum('0.0.0-sample', '1.16.0')
assert not agent_meets_minimum(None, '1.16.0')
assert not agent_meets_minimum('', '1.16.0')
# No declared floor means no opinion: an action every agent implements must
# stay available even to an endpoint whose version we cannot read.
assert agent_meets_minimum('0.0.0-sample', None)
assert agent_meets_minimum(None, None)
def test_run_script_and_update_app_are_allowlisted_delivery_actions():
"""The two software-delivery actions are allowlisted with a targetkind so
the enqueue route accepts them and carries their subject on the wire: