Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import

Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.

What arrives:

  endpoints plugin  - a ShopDB agent reporting endpoint state, telemetry,
                      services and interfaces, with a command channel the agent
                      long-polls. Ingest stays on the generic collector route
                      (ADR-006); the agent key is a UI-managed single-scope
                      token whose revocation is an instant kill switch.
  computers policies - Configuration and Business Policies (RMM-style), scope
                      trees validated at write time, actions checked against
                      the endpoints command allowlist. Feature-gated: every
                      route 404s while the setting is off, so enabling it can
                      never half-expose the surface.
  asset CSV import  - templates, a collector script carrying the site path, and
                      a validate-then-commit import. A plugin can own a virtual
                      column plus a post-write row hook, which is how a PC's
                      machine number becomes a controls link without a
                      hand-built relationships sheet.
  agent/            - the Windows agent and its installer.

Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.

MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).

Review fixes carried on top:

  - six policy GETs were jwt_required(optional=True), which authenticates a
    caller without checking anything; they require computers.view now, and six
    redundant jwt_required decorators are gone
  - the agent installer took a default BaseUrl and CollectorKey, so a mistyped
    run silently pointed an endpoint at the wrong instance; both are mandatory
    and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
  - a committing CSV import wrote no audit row

Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.

Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
This commit is contained in:
cproudlock
2026-08-28 16:17:17 -04:00
parent 93ca7587ce
commit 784964c907
149 changed files with 17545 additions and 627 deletions

View File

@@ -107,6 +107,82 @@ def authorized_service_token(scope):
return token
def mint_service_token(scope, name, owner_userid, revoke_existing=True):
"""Mint a managed service token scoped to exactly ONE permission.
The contract-surface way for a plugin to provision an unattended service
credential (an endpoint agent key, a collector key) without importing core
token internals. The returned secret is the ONLY time the full secret is
available - callers reveal it once and store only the prefix afterwards.
scope the single permission the token authorizes (e.g.
collector.ingest.endpoints). The owner must hold it.
name human label shown in the token list UI.
owner_userid the acting admin who owns (and can be audited for) the key.
revoke_existing when True (default), every currently-active token scoped to
EXACTLY this one permission is deactivated first, so
regenerating is a hard cutover: the old agent key stops
working the instant the new one is minted.
Returns (token, secret). Commits.
"""
from shopdb.core.models import User
owner = db.session.get(User, owner_userid)
if owner is None or not owner.isactive or not owner.haspermission(scope):
raise PermissionError(
'Token owner must be an active user holding ' + scope)
if revoke_existing:
for existing in list_service_tokens(scope):
existing.isactive = False
secret = ApiToken.generate_secret()
token = ApiToken(
userid=owner_userid,
name=name,
tokenprefix=ApiToken.prefix_of(secret),
tokenhash=ApiToken.hash_secret(secret),
)
token.scopelist = [scope]
db.session.add(token)
db.session.commit()
return token, secret
def list_service_tokens(scope, active_only=True):
"""Return managed tokens scoped to EXACTLY the single permission `scope`.
Used by a plugin settings page to render the current agent-key state
(prefix, created, last-used) without exposing token internals. Filters in
Python because scopes are stored as a JSON blob, not a queryable column.
"""
query = ApiToken.query
if active_only:
query = query.filter_by(isactive=True)
matches = []
for token in query.order_by(ApiToken.createddate.desc()).all():
scopelist = token.scopelist
if scopelist and len(scopelist) == 1 and scopelist[0] == scope:
matches.append(token)
return matches
def revoke_service_tokens(scope):
"""Deactivate every active managed token scoped to EXACTLY `scope`.
The kill switch: revoking the endpoint agent key stops all endpoint-agent
traffic authorized by a managed token. Returns the count revoked. Commits.
"""
revoked = 0
for token in list_service_tokens(scope):
token.isactive = False
revoked += 1
if revoked:
db.session.commit()
return revoked
def service_token_authorized(scope):
"""True when the current request carries a managed token scoped for `scope`
whose owner is active and holds that permission. Accepts X-API-Key or a