6e36df841e
Ports 37 of Benjamin Baker's commits from the wilm branch, mirrored from the
corporate GitHub at 7905917c. wilm rides the publication lineage and shares no
ancestor with wj, so this is a filtered tree patch rather than a merge:
94 files, and every path the publication export strips (docs/, tools/, mcp/,
CLAUDE.md, start-*.sh, .gitea/, the three excluded tests) is held back, because
a raw diff would delete each of them from this tree.
What comes across:
* Endpoint telemetry, phase 2. Physical disk health from two sources, NIC link
state, boot type, Defender health, processor/memory/chassis inventory, the
listening TCP set, mapped network drives and Windows Update history. Five
additive migrations, 0012 through 0016, off the 0011 head.
* Mapped drives are collected by a probe running in the interactive user's
session, because they do not exist in the agent's session 0. Two Windows
sources can disagree about one drive and the worse status wins, so a
half-broken share is never rendered as healthy.
* Windows events are reported once each, keyed on the channel's own record id,
instead of being re-sent whenever a report overlapped the last one.
* PC Detail rework: Details cards flow, the hand-written Network card gives way
to a full address history, Storage and disk health merge into one card with
two sections, Operator Reports and Corporate Tools get their own tabs.
* An opt-in restic backup overlay with a third container, restore gated on the
dump's own data revision rather than the code head, and backup/restore history.
* The agent's version is read from the binary's PE version rather than trusted
from the csproj, and the Fleet view gains an Agent Version column.
* The enqueue route now asks caller_holds instead of the owner's permissions,
which closes the scoped-token elevation on run_script, update_app and
system_restart.
Held back deliberately:
* agent/installer/install-shopdbagent.ps1 stays deleted, as upstream deleted it.
The installer that reaches a bay is generated by _install_script() in
plugins/endpoints/api/siteagent.py; the standalone script had stopped being
shipped and was collecting fixes no endpoint ever ran.
* The .gitignore hunk adding docs/*.md. Correct on the publication lineage,
where docs/ becomes the wiki and its absence is how the gates detect a
published checkout. Here it would silently ignore every new doc.
* Their deletions of docs/DEV-BUILD-TEST.md and docs/PLUGIN-HOOKS.md. Upstream
those were stubs; here PLUGIN-HOOKS.md is the 31KB document CLAUDE.md points
readers at.
* Five zero-byte files committed by an accidental `git add -A`, a run log from
the author's own machine, and docs/notes/, which is working scratch that says
so in its own header and belongs in devnotes/ on this side.
Carried with the import so it lands green: the 17 new routes are described in
docs/api-inventory.json with the spec and project map regenerated, and two new
frontend specs read fixtures through fileURLToPath(URL) which node rejects under
jsdom's own URL global.
Suite 2591 passed / 9 skipped; frontend 391 passed; naming, style and doc gates
clean.
Scope: product
251 lines
10 KiB
Python
251 lines
10 KiB
Python
"""Agent binary upload and retention routes.
|
|
|
|
Lets an operator drop in a new shopdbagent.exe or shopdb-endpoint-tray.exe from
|
|
the Endpoints page, keep the last few versions, and roll the current pointer
|
|
back to a retained build. The actual staging/retention logic lives in
|
|
agentstore.py; this module is only the HTTP surface and authorization.
|
|
|
|
Authorization boundary (deliberate):
|
|
- Uploading, listing, and selecting the current staged version is
|
|
endpoints.agentpackage - the same grant that already lets someone build and
|
|
download the site-agent ZIP. Staging a binary only changes what a FUTURE
|
|
download contains; it does not touch any running endpoint.
|
|
- Pushing that binary to live endpoints (agent replacement) is a separate,
|
|
heavier grant, endpoints.agentupdate, and is NOT wired here. It is a distinct
|
|
command family from application deploys (endpoints.deploy) so app-push rights
|
|
never imply SYSTEM-level agent replacement. That push pipe is a later slice.
|
|
|
|
Uploads are capped and PE-checked so a wrong or hostile file cannot be staged.
|
|
"""
|
|
|
|
from flask import request
|
|
|
|
from shopdb.api import (success_response, error_response, ErrorCodes, AuditLog, db,
|
|
require_permission)
|
|
|
|
from .routes import endpoints_bp
|
|
from . import agentstore
|
|
|
|
# Hard cap on an uploaded agent binary. This is a generous ceiling that still
|
|
# refuses an operator who points at a giant wrong file before it is buffered
|
|
# fully into memory-staged storage.
|
|
#
|
|
# 128 MB, not 64: the self-contained single-file tray publishes at ~69 MB
|
|
# (it carries the whole .NET runtime plus WinForms), which the old 64 MB cap
|
|
# rejected outright. The upload failed on SIZE, but the operator saw only a
|
|
# version-detection failure, because the size check returns before the PE is
|
|
# ever parsed. Config.MAX_CONTENT_LENGTH must stay above this - it bounds the
|
|
# whole multipart envelope, so a cap equal to it would reject a legal file on
|
|
# the boundary bytes alone.
|
|
_MAX_UPLOAD_BYTES = 128 * 1024 * 1024
|
|
|
|
|
|
@endpoints_bp.route('/siteagent/binaries/<binkey>', methods=['GET'])
|
|
@require_permission('endpoints.agentpackage')
|
|
def list_agent_binaries(binkey):
|
|
"""List staged versions for a binary (agent or tray), newest first."""
|
|
if not agentstore.is_valid_binary_key(binkey):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'binkey must be one of: agent, tray',
|
|
http_code=400)
|
|
return success_response({
|
|
'binary': binkey,
|
|
'current': agentstore.current_version(binkey),
|
|
'versions': agentstore.list_versions(binkey),
|
|
})
|
|
|
|
|
|
@endpoints_bp.route('/siteagent/binaries/<binkey>', methods=['POST'])
|
|
@require_permission('endpoints.agentpackage')
|
|
def upload_agent_binary(binkey):
|
|
"""Stage an uploaded binary as a new version.
|
|
|
|
Expects multipart/form-data with a 'file' upload and a 'version' field. The
|
|
version must be a strict dotted-numeric string (for example 1.6.3) so it is
|
|
safe as a directory name and orderable. The upload must be a Windows PE
|
|
executable. Older versions past the retention window are pruned, but never
|
|
the current one. By default the upload becomes the current staged build; send
|
|
makecurrent=false to stage a rollback/legacy build that joins the version
|
|
list without displacing the active current.
|
|
"""
|
|
version = (request.form.get('version') or '').strip()
|
|
makecurrent = (request.form.get('makecurrent') or 'true').strip().lower()
|
|
makecurrent = makecurrent not in ('false', '0', 'no')
|
|
if not agentstore.is_valid_binary_key(binkey):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'binkey must be one of: agent, tray',
|
|
http_code=400)
|
|
|
|
upload = request.files.get('file')
|
|
if upload is None or not upload.filename:
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'a file upload is required',
|
|
http_code=400)
|
|
|
|
data = upload.read(_MAX_UPLOAD_BYTES + 1)
|
|
if len(data) > _MAX_UPLOAD_BYTES:
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'uploaded binary exceeds the size limit',
|
|
http_code=400)
|
|
if not agentstore.looks_like_pe(data):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'uploaded file is not a Windows executable',
|
|
http_code=400)
|
|
|
|
# Gate the version at the PE, not at the raw input. Product versions carry
|
|
# semver build metadata (1.5.0+abcdef...) that is not a legal directory
|
|
# name, so the operator's typed string is only a hint. The dotted-numeric
|
|
# FILE version stamped in the binary's PE resource is the authority; fall
|
|
# back to it whenever the typed value is missing or not dotted-numeric, and
|
|
# only reject when neither yields a usable version.
|
|
if not agentstore.is_valid_version(version):
|
|
detected = agentstore.read_pe_file_version(data)
|
|
if agentstore.is_valid_version(detected):
|
|
version = detected
|
|
else:
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'version must be a dotted-numeric string such as 1.6.3',
|
|
http_code=400)
|
|
|
|
result = agentstore.store_version(binkey, version, data,
|
|
make_current=makecurrent)
|
|
|
|
# This is the binary every future package installs and runs as SYSTEM on
|
|
# the fleet. Without a record, an incident has no answer to "who staged
|
|
# this, and when" - the manifest holds the file, not the person. The digest
|
|
# identifies the BYTES, because a version string is chosen by whoever
|
|
# uploads and re-uploading one replaces the file in place.
|
|
entry = next((v for v in result['versions']
|
|
if v.get('version') == version), {})
|
|
AuditLog.log(
|
|
'uploaded', 'AgentBinary', entityname='%s %s' % (binkey, version),
|
|
changes={'sha256': entry.get('sha256'), 'size': entry.get('size'),
|
|
'madecurrent': makecurrent})
|
|
db.session.commit()
|
|
|
|
return success_response({
|
|
'binary': binkey,
|
|
'current': result['current'],
|
|
'versions': result['versions'],
|
|
})
|
|
|
|
|
|
@endpoints_bp.route('/siteagent/binaries/<binkey>/detect', methods=['POST'])
|
|
@require_permission('endpoints.agentpackage')
|
|
def detect_agent_binary_version(binkey):
|
|
"""Read the PE file version of an uploaded EXE without staging it.
|
|
|
|
The Build Site Agent page calls this on file-select so the version field is
|
|
prefilled from the binary's own version resource - the operator does not
|
|
have to read the EXE properties by hand for a current-release upload.
|
|
Returns {version} which is null when the file carries no readable version.
|
|
"""
|
|
if not agentstore.is_valid_binary_key(binkey):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'binkey must be one of: agent, tray',
|
|
http_code=400)
|
|
|
|
upload = request.files.get('file')
|
|
if upload is None or not upload.filename:
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'a file upload is required',
|
|
http_code=400)
|
|
|
|
data = upload.read(_MAX_UPLOAD_BYTES + 1)
|
|
if len(data) > _MAX_UPLOAD_BYTES:
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'uploaded binary exceeds the size limit',
|
|
http_code=400)
|
|
if not agentstore.looks_like_pe(data):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'uploaded file is not a Windows executable',
|
|
http_code=400)
|
|
|
|
return success_response({
|
|
'binary': binkey,
|
|
'version': agentstore.read_pe_file_version(data),
|
|
})
|
|
|
|
|
|
@endpoints_bp.route('/siteagent/binaries/<binkey>/adopt', methods=['POST'])
|
|
@require_permission('endpoints.agentpackage')
|
|
def adopt_legacy_agent_binary(binkey):
|
|
"""Adopt the unmanaged legacy flat file into the version store.
|
|
|
|
The pre-versioning install dropped a single shopdbagent.exe (or tray exe)
|
|
straight under siteagent/. This copies that file in as a real staged version
|
|
so it shows a version and joins retention, without the operator having to
|
|
re-upload it. The version comes from the request when the binary carries no
|
|
readable PE version resource; otherwise the detected version is used.
|
|
"""
|
|
if not agentstore.is_valid_binary_key(binkey):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'binkey must be one of: agent, tray',
|
|
http_code=400)
|
|
|
|
status = agentstore.legacy_status(binkey)
|
|
if status is None:
|
|
return error_response(
|
|
ErrorCodes.NOT_FOUND,
|
|
'there is no unmanaged legacy binary to adopt',
|
|
http_code=404)
|
|
|
|
source = request.get_json(silent=True) or {}
|
|
version = (source.get('version') or status.get('version') or '').strip()
|
|
if not agentstore.is_valid_version(version):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'version must be a dotted-numeric string such as 1.6.3',
|
|
http_code=400)
|
|
|
|
result = agentstore.adopt_legacy(binkey, version)
|
|
if result is None:
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'the legacy binary could not be adopted',
|
|
http_code=400)
|
|
return success_response({
|
|
'binary': binkey,
|
|
'current': result['current'],
|
|
'versions': result['versions'],
|
|
})
|
|
|
|
|
|
@endpoints_bp.route('/siteagent/binaries/<binkey>/current', methods=['PUT'])
|
|
@require_permission('endpoints.agentpackage')
|
|
def set_current_agent_binary(binkey):
|
|
"""Point 'current' at an already-staged version (roll back or forward)."""
|
|
if not agentstore.is_valid_binary_key(binkey):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'binkey must be one of: agent, tray',
|
|
http_code=400)
|
|
source = request.get_json(silent=True) or {}
|
|
version = (source.get('version') or '').strip()
|
|
if not agentstore.is_valid_version(version):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'version must be a dotted-numeric string such as 1.6.3',
|
|
http_code=400)
|
|
if not agentstore.set_current(binkey, version):
|
|
return error_response(
|
|
ErrorCodes.NOT_FOUND,
|
|
'that version is not staged',
|
|
http_code=404)
|
|
return success_response({
|
|
'binary': binkey,
|
|
'current': agentstore.current_version(binkey),
|
|
'versions': agentstore.list_versions(binkey),
|
|
})
|