Make main the lean product a site receives

main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.

WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.

WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.

A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:

deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.

agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.

DECISIONS THIS ENCODES, rather than defers:

- The plugin-authoring lab builds printedparts, whose manifest depends on
  employees, so it could not stay. The lab remains on the site line and main
  ships no build-along tutorial. That is a real loss for an adopting developer
  and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
  Each splits one notification into a card per employee and renders a photo,
  which needs a plugin this branch does not bundle. Seeding them would offer an
  author a type that cannot resolve anybody. A site that installs employees gets
  them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
  with Docker, because that is what a receiving site gets.

The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.

Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.
This commit is contained in:
cproudlock
2026-09-19 11:24:40 -04:00
parent cb3975049c
commit 534f179ee6
675 changed files with 115759 additions and 60239 deletions

View File

@@ -4,6 +4,7 @@ import glob
import hashlib
import re
import os
import shutil
from flask import Blueprint, request, current_app
from flask_jwt_extended import jwt_required
@@ -36,11 +37,147 @@ APP_PACKAGE_EXTENSIONS = {'.exe', '.msi', '.msp', '.zip', '.7z', '.cab', '.iso',
'.appx', '.msix', '.ps1', '.bat', '.txt', '.pdf'}
APP_PACKAGE_URL_PREFIX = '/api/applications/package/'
def _requires_policy_ack(app, data):
"""True when this save would newly generate an install-enforcing policy.
Ticking "Required on all PCs" does not just flag a report row: it creates a
policy that installs this application on every matched endpoint. That is a
real change to real machines, so the caller has to acknowledge it
explicitly. Enforced here rather than only in the form because the checkbox
is reachable through the API too, and a confirmation dialog that can be
skipped by calling the endpoint directly is not a safeguard.
Only NEWLY ticking it needs acknowledgement. Unticking, or saving an app
that was already required, does not.
"""
if not data.get('isrequired'):
return False
if app is not None and app.isrequired:
return False
return not data.get('acknowledgepolicy')
POLICY_ACK_MESSAGE = (
'Marking this application as required on all PCs creates a configuration '
'policy that installs it on every non-server endpoint it matches, once that '
'policy is added to an active business policy. It is not added to one '
'automatically. Confirm to create the policy.'
)
# Returned alongside policyresult='created'. The generated policy is not
# attached to any BusinessPolicy, and the resolver only applies policies reached
# through one, so saying "this will install everywhere" would be untrue.
POLICY_CREATED_NOTE = (
'A configuration policy was created for this application, but it is not '
'attached to any business policy and installs nothing until an admin adds '
'it to one.'
)
# Deleting an application is irreversible and takes its versions and uploaded
# installer with it, so the caller has to type this rather than click once.
# Checked server-side as well as in the dialog: the endpoint is reachable
# directly, and a confirmation that only exists in the UI is not a safeguard.
DELETE_CONFIRM_TOKEN = 'OK'
DELETE_CONFIRM_MESSAGE = (
'Deleting an application permanently removes it, its versions, and its '
'uploaded installer. This cannot be undone. Type OK to confirm.'
)
def _delete_confirmed(data):
"""True when the caller typed the confirmation token exactly.
Compared case-sensitively after stripping surrounding whitespace. Accepting
"ok" would defeat the point of asking for deliberate input.
"""
if not isinstance(data, dict):
return False
return str(data.get('confirm', '')).strip() == DELETE_CONFIRM_TOKEN
def _validate_monitorable(data, existing=None):
"""Return an error message if monitorable is set without its two fields.
A process monitor needs an image name to watch for and a path to launch when
it is missing; without both, ticking the box would create a monitor that
cannot do anything. Enforced here rather than as a DB constraint so the
columns can stay nullable for rows that predate the feature.
`existing` lets a PATCH-style update validate the MERGED result: a payload
that only toggles the checkbox is still checked against values already
stored on the row.
"""
def merged(field):
if field in data:
return (data.get(field) or '').strip()
return ((getattr(existing, field, None) or '').strip()
if existing is not None else '')
if 'ismonitorable' in data:
monitorable = data.get('ismonitorable')
else:
monitorable = getattr(existing, 'ismonitorable', False)
if not monitorable:
return None
if not merged('monitorlaunchpath'):
return 'A monitorable application needs a launch path'
if not merged('monitorprocessname'):
return 'A monitorable application needs a process name'
return None
def _policy_payload(payload, policyresult):
"""Attach the policy sync outcome (and the not-attached note) to a response."""
if policyresult in ('created', 'removed', 'kept'):
payload['policyresult'] = policyresult
if policyresult == 'created':
payload['policynote'] = POLICY_CREATED_NOTE
return payload
def _deactivate_policies_for_app(app_id):
"""Deactivate remaining policies that reference a deleted appid, if available."""
try:
from plugins.computers.services.apppolicy import deactivate_policies_for_app
except ImportError:
return []
return deactivate_policies_for_app(app_id)
def _sync_required_app_policy(app):
"""Sync the auto-generated "required on all PCs" policy, if available.
The policy engine lives in the computers PLUGIN, which may be absent or
disabled, so this is a guarded import: core must not hard-depend on a
plugin. When the plugin is not installed the checkbox keeps its original
compliance-report meaning and nothing else happens.
"""
try:
from plugins.computers.services.apppolicy import sync_required_app_policy
except ImportError:
return None
try:
return sync_required_app_policy(app)
except Exception:
# A policy-generation failure must not lose the user's application edit.
current_app.logger.exception(
'Failed to sync the required-app policy for appid=%s', app.appid)
return None
# A real installer runs to hundreds of megabytes; an ISO runs to gigabytes and
# does not belong in an instance directory. 500MB is the line: enough for the
# msi/exe this is for, small enough that the disk cannot vanish behind it.
MAX_PACKAGE_BYTES = 500 * 1024 * 1024
# How much of an installer to read when looking for its version resource. The
# PE resource directory is usually within the first few MB; reading the whole
# file would mean holding a 500MB upload in memory to learn a version string.
PE_VERSION_SCAN_BYTES = 32 * 1024 * 1024
def _appimage_dir():
return os.path.join(current_app.instance_path, 'applicationimages')
@@ -64,6 +201,46 @@ def _sha256_of(stream):
return digest.hexdigest()
def _detect_installer_version(stream):
"""Read the FileVersion out of an uploaded PE installer, or None.
Deliberately reuses the endpoints plugin's PE reader rather than parsing the
format again here. A version detected at upload has to match what the agent
reports from the installed machine, and two separate implementations of the
same structure walk would drift apart and make an endpoint look
out-of-compliance for no real reason.
FileVersion only, never ProductVersion: ProductVersion may carry semver
metadata (1.5.0+abc123) which is not a version number we can compare or use
in a path. Returns None for MSI and archives, which are not PE files at all
and simply have no version to read.
"""
try:
from plugins.endpoints.api.agentstore import read_pe_file_version
except ImportError:
return None
position = stream.tell()
try:
# The version resource lives in the PE header near the start, but the
# resource directory it points into can sit much deeper. Read a bounded
# prefix so a 500MB installer is not pulled into memory whole.
data = stream.read(PE_VERSION_SCAN_BYTES)
finally:
stream.seek(position)
# The parser documents itself as never raising, but it walks offsets read
# out of an attacker-supplied file. A version is a convenience; it must not
# cost the operator the upload of a 500MB installer if that guarantee ever
# slips.
try:
return read_pe_file_version(data)
except Exception:
current_app.logger.warning('Version detection failed for an installer',
exc_info=True)
return None
def _replace_upload(directory, stem, ext, upload):
"""Save one file per application, replacing any prior extension.
@@ -212,6 +389,13 @@ def create_application():
http_code=409
)
monitor_error = _validate_monitorable(data)
if monitor_error:
return error_response(ErrorCodes.VALIDATION_ERROR, monitor_error)
if _requires_policy_ack(None, data):
return error_response(ErrorCodes.VALIDATION_ERROR, POLICY_ACK_MESSAGE)
app = Application(
appname=data['appname'],
appdescription=data.get('appdescription'),
@@ -225,6 +409,9 @@ def create_application():
isprinter=data.get('isprinter', False),
islicenced=data.get('islicenced', False),
isrequired=data.get('isrequired', False),
ismonitorable=data.get('ismonitorable', False),
monitorlaunchpath=data.get('monitorlaunchpath'),
monitorprocessname=data.get('monitorprocessname'),
image=data.get('image')
)
@@ -234,9 +421,13 @@ def create_application():
AuditLog.log('created', 'Application', entityid=app.appid, entityname=app.appname)
policyresult = _sync_required_app_policy(app)
db.session.commit()
return success_response(app.to_dict(), message='Application created', http_code=201)
payload = _policy_payload(app.to_dict(), policyresult)
return success_response(payload, message='Application created', http_code=201)
@applications_bp.route('/<int:app_id>', methods=['PUT'])
@@ -260,12 +451,34 @@ def update_application(app_id: int):
http_code=409
)
monitor_error = _validate_monitorable(data, existing=app)
if monitor_error:
return error_response(ErrorCodes.VALIDATION_ERROR, monitor_error)
if _requires_policy_ack(app, data):
return error_response(ErrorCodes.VALIDATION_ERROR, POLICY_ACK_MESSAGE)
fields = [
'appname', 'appdescription', 'supportteamid', 'isinstallable',
'applicationnotes', 'installpath', 'applicationlink', 'documentationpath',
'ishidden', 'isprinter', 'islicenced', 'isrequired', 'image', 'isactive'
'ishidden', 'isprinter', 'islicenced', 'isrequired', 'image', 'isactive',
'ismonitorable', 'monitorlaunchpath', 'monitorprocessname'
]
# Only touch policy when the checkbox itself moved. Syncing on every save
# would let an unrelated edit resurrect a policy an admin had deleted.
#
# Unticking is synced BEFORE any field is applied, so a PUT that renames and
# unticks in one go withdraws the policy that belongs to the app as it was,
# not one looked up under the new name.
policyresult = None
unticking = ('isrequired' in data and bool(app.isrequired)
and not data['isrequired'])
if unticking:
app.isrequired = False
policyresult = _sync_required_app_policy(app)
app.isrequired = True
changes = {}
for key in fields:
if key in data:
@@ -279,27 +492,210 @@ def update_application(app_id: int):
AuditLog.log('updated', 'Application', entityid=app.appid,
entityname=app.appname, changes=changes)
if 'isrequired' in changes and not unticking:
policyresult = _sync_required_app_policy(app)
elif 'appname' in changes and app.isrequired:
# A rename of a required app keeps its generated policy's name current.
_sync_required_app_policy(app)
apply_import_timestamps(app, data)
db.session.commit()
return success_response(app.to_dict(), message='Application updated')
payload = _policy_payload(app.to_dict(), policyresult)
return success_response(payload, message='Application updated')
def _remove_package_files(app_id: int):
"""Delete the stored installer(s) for an application, leaving the image.
Split out from _remove_upload_files because revert-to-record removes the
executables but keeps the icon: the record stays in the catalogue and a
record with no image is a worse record. Delete calls both.
Best-effort by design: a missing or unreadable file must not abort the
operation and strand the row. Returns the filenames actually removed so the
audit entry records what went.
"""
removed = []
packagedir = os.path.join(_apppackage_dir(), str(app_id))
for path in glob.glob(os.path.join(packagedir, '*')):
removed.append(os.path.basename(path))
if os.path.isdir(packagedir):
shutil.rmtree(packagedir, ignore_errors=True)
if os.path.isdir(packagedir):
current_app.logger.warning(
'Could not remove %s for appid=%s', packagedir, app_id)
return removed
def _remove_upload_files(app_id: int):
"""Delete the stored installer and image for an application.
Used by delete, which takes everything. Revert calls
_remove_package_files() instead so the image survives.
"""
removed = []
# The image is still one flat file per application; the installer now lives
# in a per-application directory so it can keep its vendor filename.
pattern = os.path.join(_appimage_dir(),
secure_filename(f'application-{app_id}') + '.*')
for path in glob.glob(pattern):
try:
os.remove(path)
removed.append(os.path.basename(path))
except OSError:
current_app.logger.warning(
'Could not remove %s while deleting appid=%s', path, app_id)
return removed + _remove_package_files(app_id)
def _clear_application_references(app_id: int):
"""Detach rows that point at this application before it is deleted.
The FKs differ in intent, so they are not treated the same:
- appversions.appid is NOT NULL and the versions are meaningless without
the application, so they go.
- computerinstalledapps.appid is NOT NULL; these are inventory links, so
the link goes while the computer stays.
- knowledgebase.appid is nullable and the article is somebody's writing,
so it is unlinked rather than deleted.
"""
AppVersion.query.filter_by(appid=app_id).delete(synchronize_session=False)
models = _computer_models()
if models is not None:
_, ComputerInstalledApp = models
ComputerInstalledApp.query.filter_by(appid=app_id).delete(
synchronize_session=False)
try:
from plugins.knowledgebase.models import KnowledgeBase
except ImportError:
return
KnowledgeBase.query.filter_by(appid=app_id).update(
{'appid': None}, synchronize_session=False)
@applications_bp.route('/<int:app_id>', methods=['DELETE'])
@require_permission('applications.delete')
def delete_application(app_id: int):
"""Delete (deactivate) an application."""
"""Permanently delete an application, its versions, and its uploads.
This is a hard delete. The previous soft delete only set isactive=False,
which left the name reserved forever: creating an application with the same
name failed with "already exists" against a row no screen would show.
"""
app = db.session.get(Application, app_id)
if not app:
return error_response(ErrorCodes.NOT_FOUND, 'Application not found', http_code=404)
app.isactive = False
if not _delete_confirmed(request.get_json(silent=True)):
return error_response(ErrorCodes.VALIDATION_ERROR, DELETE_CONFIRM_MESSAGE)
AuditLog.log('deleted', 'Application', entityid=app.appid, entityname=app.appname)
# Withdraw the generated policy first. A policy that outlived its
# application would keep installing something that no longer exists.
appname = app.appname
app.isrequired = False
policyresult = _sync_required_app_policy(app)
# 'kept' means a customised generated policy survived the sync, and other
# hand-authored policies may name this appid too. Every one of them would
# keep trying to install an application that no longer exists, so they are
# deactivated (not deleted - they are someone's work) and reported.
deactivated = _deactivate_policies_for_app(app_id)
_clear_application_references(app_id)
removed = _remove_upload_files(app_id)
db.session.delete(app)
# The audit row is the only record left that this application existed, so
# it names the files that went with it.
auditchanges = {}
if removed:
auditchanges['removedfiles'] = removed
if deactivated:
auditchanges['deactivatedpolicies'] = deactivated
AuditLog.log('deleted', 'Application', entityid=app_id, entityname=appname,
changes=auditchanges or None)
db.session.commit()
return success_response(message='Application deleted')
payload = {}
if policyresult in ('removed', 'kept'):
payload['policyresult'] = policyresult
if deactivated:
payload['deactivatedpolicies'] = deactivated
message = 'Application deleted'
if deactivated:
message += ('; deactivated {} policy(ies) that referenced it: {}'
.format(len(deactivated), ', '.join(deactivated)))
return success_response(payload or None, message=message)
REVERT_CONFIRM_MESSAGE = (
'Remove all policy associations and uploaded executables? This action '
"cannot be reversed. Type OK to proceed"
)
@applications_bp.route('/<int:app_id>/revert', methods=['POST'])
@require_permission('applications.edit')
def revert_application_to_record(app_id: int):
"""Strip an application back to a catalogue record.
The middle ground between editing and deleting. Deleting loses the name,
the description and the knowledgebase links, which is too much when the
intent is only to stop deploying something and re-upload a clean installer.
This withdraws the generated policy, drops the version rows and removes the
uploaded files, while the application itself stays in the catalogue.
Inventory links are deliberately kept: computerinstalledapps records that a
machine really does have this software on it, and that stays true whether
or not we still ship an installer for it. Deleting those would make the
fleet look clean when it is not.
Gated on applications.edit rather than applications.delete: the record
survives, and a user trusted to change an application is trusted to stop
deploying it.
"""
app = db.session.get(Application, app_id)
if not app:
return error_response(ErrorCodes.NOT_FOUND, 'Application not found',
http_code=404)
if not _delete_confirmed(request.get_json(silent=True)):
return error_response(ErrorCodes.VALIDATION_ERROR, REVERT_CONFIRM_MESSAGE)
# Withdraw the generated policy first, for the same reason delete does: a
# policy left behind would keep pushing an installer we just removed.
app.isrequired = False
_sync_required_app_policy(app)
AppVersion.query.filter_by(appid=app_id).delete(synchronize_session=False)
# Executables only. The image is not an executable and the warning does not
# offer to remove it - the record survives this operation, and stripping its
# icon would degrade the thing we are explicitly keeping.
removed = _remove_package_files(app_id)
# Only clear an installpath we own. A UNC path typed by an operator points
# at their share, not our upload directory, and is not ours to wipe.
if (app.installpath or '').startswith(APP_PACKAGE_URL_PREFIX):
app.installpath = None
app.installersha256 = None
AuditLog.log('updated', 'Application', entityid=app_id,
entityname=app.appname,
changes={'revertedtorecord': {'new': True},
'removedfiles': removed})
db.session.commit()
return success_response(app.to_dict(), message='Application reverted to record')
# ---- Versions ----
@@ -613,14 +1009,43 @@ def delete_application_image(app_id: int):
return success_response(app.to_dict(), message='Application image removed')
def _store_package(app_id: int, upload):
"""Save an installer under its own vendor filename, one per application.
Keeping the shipped name matters at import: "DeviceLoggerSetup.msi" is
recognisable as the thing already in the catalogue, whereas the old
application-<id>.msi told the operator nothing and made a re-import look
like new software, which is how duplicate applications and versions got
created in the first place.
Files are namespaced by application id so two applications shipping a
generic "setup.exe" cannot overwrite each other. Any previous installer for
this application is removed first, so replacing one does not accumulate.
Returns (storedname, originalname).
"""
directory = os.path.join(_apppackage_dir(), str(app_id))
if os.path.isdir(directory):
shutil.rmtree(directory, ignore_errors=True)
os.makedirs(directory, exist_ok=True)
original = os.path.basename(upload.filename or '')
# secure_filename can reduce a name to nothing (all-unicode, all-dots), and
# an empty name would save as a bare directory entry.
stored = secure_filename(original) or 'installer{}'.format(
os.path.splitext(original)[1].lower())
upload.save(os.path.join(directory, stored))
return stored, original
@applications_bp.route('/<int:app_id>/package', methods=['POST'])
@require_permission('applications.edit')
def upload_application_package(app_id: int):
"""Upload (or replace) the installer for an application.
multipart/form-data: file=<installer>. The original filename is kept in
installpath so the download arrives named the way the vendor shipped it,
and the stored copy is application-<id><ext>.
multipart/form-data: file=<installer>. The installer is stored under the
filename the vendor shipped, inside a per-application directory.
"""
app = db.session.get(Application, app_id)
if not app:
@@ -650,8 +1075,11 @@ def upload_application_package(app_id: int):
'on the share and link to them with Install Path.'.format(
size / 1048576, MAX_PACKAGE_BYTES / 1048576))
filename = _replace_upload(_apppackage_dir(), f'application-{app_id}', ext, upload)
app.installpath = '{}{}'.format(APP_PACKAGE_URL_PREFIX, filename)
# Read the version before saving: the stream is consumed by save().
detectedversion = _detect_installer_version(upload.stream)
filename, _original = _store_package(app_id, upload)
app.installpath = '{}{}/{}'.format(APP_PACKAGE_URL_PREFIX, app_id, filename)
# The endpoint installer push pins bytes by digest; compute it once here so
# the server, not the agent, decides what the correct payload is. Stamp it
@@ -682,6 +1110,10 @@ def upload_application_package(app_id: int):
return success_response(
{**app.to_dict(), 'uploadedfilename': upload.filename, 'bytes': size,
'sha256': sha256,
# None means "no version resource" - the form must ask the operator
# rather than silently saving an installer with no version, which is
# how duplicate version rows appeared.
'detectedversion': detectedversion,
'appversionid': stamped_version.appversionid if stamped_version else None},
message='Installer uploaded')
@@ -698,6 +1130,10 @@ def serve_application_package(filename):
Gated on applications.download, not applications.view: a guest may be
granted the metadata permission, and "an open URL would publish it" is just
as true of a URL any anonymous visitor can reach.
filename is "<appid>/<vendorname>" now that installers keep the name they
shipped under. send_from_directory refuses to escape the base directory, so
the appid segment cannot be walked out of with "..".
"""
return send_upload(_apppackage_dir(), filename, as_attachment=True)
@@ -712,12 +1148,12 @@ def delete_application_package(app_id: int):
f'Application with ID {app_id} not found',
http_code=404)
for old in glob.glob(os.path.join(_apppackage_dir(),
secure_filename(f'application-{app_id}') + '.*')):
os.remove(old)
shutil.rmtree(os.path.join(_apppackage_dir(), str(app_id)),
ignore_errors=True)
# Only clear the path if it pointed at the upload; a site's own share path
# was typed by a person and is not ours to wipe.
if (app.installpath or '').startswith(APP_PACKAGE_URL_PREFIX):
app.installpath = None
app.installersha256 = None
db.session.commit()
return success_response(app.to_dict(), message='Installer removed')