Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import

Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.

What arrives:

  endpoints plugin  - a ShopDB agent reporting endpoint state, telemetry,
                      services and interfaces, with a command channel the agent
                      long-polls. Ingest stays on the generic collector route
                      (ADR-006); the agent key is a UI-managed single-scope
                      token whose revocation is an instant kill switch.
  computers policies - Configuration and Business Policies (RMM-style), scope
                      trees validated at write time, actions checked against
                      the endpoints command allowlist. Feature-gated: every
                      route 404s while the setting is off, so enabling it can
                      never half-expose the surface.
  asset CSV import  - templates, a collector script carrying the site path, and
                      a validate-then-commit import. A plugin can own a virtual
                      column plus a post-write row hook, which is how a PC's
                      machine number becomes a controls link without a
                      hand-built relationships sheet.
  agent/            - the Windows agent and its installer.

Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.

MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).

Review fixes carried on top:

  - six policy GETs were jwt_required(optional=True), which authenticates a
    caller without checking anything; they require computers.view now, and six
    redundant jwt_required decorators are gone
  - the agent installer took a default BaseUrl and CollectorKey, so a mistyped
    run silently pointed an endpoint at the wrong instance; both are mandatory
    and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
  - a committing CSV import wrote no audit row

Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.

Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.

plugins/computers/services/__init__.py

@@ -0,0 +1 @@
+"""Computers plugin services: pure policy logic (scope + resolution)."""

plugins/computers/services/policyresolver.py

@@ -0,0 +1,168 @@
+"""Effective-policy resolution: the single swappable policy entry point.
+
+This module answers one question for the rest of the system: "given an endpoint,
+what actions apply and how?" It is the boundary the endpoints allowlist seam
+calls - never the policy tables directly - so the policy ASSEMBLY ENGINE can be
+swapped without touching enforcement. Today it folds the local computers-plugin
+tables; a future GE-Enforce integration can replace build_effective_policy (or
+feed it) and every consumer keeps working unchanged.
+
+Resolution model (per the design):
+  - Build an endpoint context (PC type + installed appids) from the Computer row.
+  - Walk every active Business Policy in priority order (lower first, ties by id).
+  - Within each, walk its Configuration Policies in sortorder.
+  - A Configuration Policy contributes only if its scope matches the endpoint.
+  - For each action name, FIRST MATCH WINS: once an action is decided it is not
+    overridden by a later, lower-priority policy.
+
+The result is an EffectivePolicy: a flat, resolved map of action name -> decision
+(mode desired/gate, plus desiredstate/params), which is exactly what enforcement
+and the allowlist need.
+"""
+
+from .policyscope import evaluate_scope
+
+
+class EffectiveAction:
+    """One resolved action decision for an endpoint."""
+
+    __slots__ = ('action', 'mode', 'desiredstate', 'params', 'intervalminutes',
+                 'businesspolicyid', 'configurationpolicyid')
+
+    def __init__(self, action, mode, desiredstate, params, intervalminutes,
+                 businesspolicyid, configurationpolicyid):
+        self.action = action
+        self.mode = mode
+        self.desiredstate = desiredstate
+        self.params = params
+        self.intervalminutes = intervalminutes
+        self.businesspolicyid = businesspolicyid
+        self.configurationpolicyid = configurationpolicyid
+
+    def to_dict(self):
+        return {
+            'action': self.action,
+            'mode': self.mode,
+            'desiredstate': self.desiredstate,
+            'params': self.params,
+            'intervalminutes': self.intervalminutes,
+            'businesspolicyid': self.businesspolicyid,
+            'configurationpolicyid': self.configurationpolicyid,
+        }
+
+
+class EffectivePolicy:
+    """The folded, resolved policy for a single endpoint.
+
+    actions maps action name -> EffectiveAction (first-match-wins already
+    applied). Convenience views expose the gate allowlist and desired-state set
+    that enforcement and the endpoints seam consume.
+    """
+
+    def __init__(self, actions):
+        self.actions = actions
+
+    def allowed_actions(self):
+        """Set of action names permitted on this endpoint (gate + desired)."""
+        return set(self.actions.keys())
+
+    def desired_actions(self):
+        """action name -> EffectiveAction for mode == 'desired' only."""
+        return {name: eff for name, eff in self.actions.items()
+                if eff.mode == 'desired'}
+
+    def to_dict(self):
+        return {name: eff.to_dict() for name, eff in self.actions.items()}
+
+
+def build_endpoint_context(computer):
+    """Resolve a Computer row into the context the scope evaluator needs.
+
+    Returns {'computertypeid': int|None, 'appids': set[int]}.
+    """
+    appids = set()
+    for link in getattr(computer, 'installedapps', None) or ():
+        if getattr(link, 'isactive', True) and link.appid is not None:
+            appids.add(link.appid)
+    return {
+        'computertypeid': getattr(computer, 'computertypeid', None),
+        'appids': appids,
+    }
+
+
+def _iter_matching_configpolicies(business_policies, context):
+    """Yield (businesspolicyid, ConfigurationPolicy) in deterministic order.
+
+    Business Policies by (priority, businesspolicyid); their items by sortorder.
+    Only active policies whose scope matches the context are yielded.
+    """
+    ordered_bp = sorted(
+        business_policies,
+        key=lambda bp: (bp.priority, bp.businesspolicyid),
+    )
+    for business_policy in ordered_bp:
+        if not business_policy.isactive:
+            continue
+        for item in sorted(business_policy.items,
+                           key=lambda it: (it.sortorder, it.businesspolicyitemid)):
+            config_policy = item.configurationpolicy
+            if config_policy is None or not config_policy.isactive:
+                continue
+            if evaluate_scope(config_policy.scopejson, context):
+                yield business_policy.businesspolicyid, config_policy
+
+
+def build_effective_policy(computer, business_policies):
+    """Fold matching policies into an EffectivePolicy for one endpoint.
+
+    computer: the Computer row. business_policies: iterable of BusinessPolicy
+    (with items/configurationpolicy loaded). Pure given its inputs so it stays
+    unit-testable and reusable by an alternate assembly engine.
+    """
+    context = build_endpoint_context(computer)
+    resolved = {}
+    for businesspolicyid, config_policy in _iter_matching_configpolicies(
+            business_policies, context):
+        for action_row in config_policy.actions:
+            if not action_row.isactive:
+                continue
+            name = action_row.action
+            if name in resolved:
+                # First match wins - a higher-priority policy already decided
+                # this action; later policies do not override it.
+                continue
+            resolved[name] = EffectiveAction(
+                action=name,
+                mode=action_row.mode,
+                desiredstate=action_row.desiredstate,
+                params=action_row.params,
+                intervalminutes=action_row.intervalminutes,
+                businesspolicyid=businesspolicyid,
+                configurationpolicyid=config_policy.configurationpolicyid,
+            )
+    return EffectivePolicy(resolved)
+
+
+def effective_policy_for(assetid):
+    """Resolve the effective policy for an endpoint by its assetid.
+
+    This is the public entry point the endpoints plugin calls. It returns an
+    empty EffectivePolicy when the feature is off or the asset is not a known
+    computer, so callers get a safe "no policy" answer without special-casing.
+
+    NOTE for GE-Enforce interop: this function is the seam. To hand policy
+    assembly to GE-Enforce later, replace the body below (or dispatch on source)
+    while keeping this signature and the EffectivePolicy return type intact.
+    """
+    from ..policyflag import policy_flag_enabled
+    if not policy_flag_enabled():
+        return EffectivePolicy({})
+
+    from ..models import Computer, BusinessPolicy
+
+    computer = Computer.query.filter_by(assetid=assetid).first()
+    if computer is None:
+        return EffectivePolicy({})
+
+    business_policies = BusinessPolicy.query.filter_by(isactive=True).all()
+    return build_effective_policy(computer, business_policies)

plugins/computers/services/policyscope.py

@@ -0,0 +1,96 @@
+"""Boolean scope evaluation for Configuration Policies.
+
+A policy's scopejson is a small typed boolean AST over two endpoint dimensions,
+PC type and installed application. This module is the ONLY place that interprets
+that AST: it validates the shape (known node types, bounded depth) and evaluates
+a validated tree against a resolved endpoint context.
+
+Node grammar (all dict nodes carry a "type"):
+    {"type": "and",    "nodes": [<node>, ...]}
+    {"type": "or",     "nodes": [<node>, ...]}
+    {"type": "not",    "node": <node>}
+    {"type": "pctype", "id": <computertypeid:int>}
+    {"type": "app",    "id": <appid:int>}
+    {"type": "true"}   # matches everything (an unscoped/global policy)
+
+Kept pure and dependency-free (no db, no Flask) so it is trivially unit-tested
+and so a future GE-Enforce assembly engine can reuse the exact same evaluator on
+the exact same AST when it composes policies.
+"""
+
+# Bound the tree so a hand-crafted or imported AST cannot cause pathological
+# recursion. Ten levels is far past any policy a human would author.
+MAX_SCOPE_DEPTH = 10
+
+_LOGICAL = {'and', 'or', 'not'}
+_LEAF = {'pctype', 'app', 'true'}
+
+
+class ScopeError(ValueError):
+    """Raised when a scope AST is malformed. Carries a human-readable reason."""
+
+
+def validate_scope(node, _depth=0):
+    """Validate a scope AST, raising ScopeError on any problem.
+
+    Returns None on success. Call this at write time (CRUD) so a bad tree never
+    reaches the database; evaluate_scope then trusts the stored shape.
+    """
+    if _depth > MAX_SCOPE_DEPTH:
+        raise ScopeError(f'scope nested deeper than {MAX_SCOPE_DEPTH} levels')
+    if not isinstance(node, dict):
+        raise ScopeError('scope node must be an object')
+
+    ntype = node.get('type')
+    if ntype not in _LOGICAL and ntype not in _LEAF:
+        raise ScopeError(f'unknown scope node type: {ntype!r}')
+
+    if ntype in ('and', 'or'):
+        nodes = node.get('nodes')
+        if not isinstance(nodes, list) or not nodes:
+            raise ScopeError(f'{ntype} requires a non-empty "nodes" list')
+        for child in nodes:
+            validate_scope(child, _depth + 1)
+    elif ntype == 'not':
+        child = node.get('node')
+        if child is None:
+            raise ScopeError('not requires a "node"')
+        validate_scope(child, _depth + 1)
+    elif ntype in ('pctype', 'app'):
+        ident = node.get('id')
+        if not isinstance(ident, int) or isinstance(ident, bool):
+            raise ScopeError(f'{ntype} requires an integer "id"')
+    # 'true' carries no payload.
+
+
+def evaluate_scope(node, context):
+    """Evaluate a validated scope AST against an endpoint context.
+
+    context is expected to expose:
+        context['computertypeid'] -> int or None
+        context['appids']         -> set/collection of installed appids
+
+    Returns True when the endpoint falls in scope. An empty/None scope means the
+    policy is unscoped and matches everything (same as {"type": "true"}), which
+    keeps a freshly-created policy usable before a scope is drawn.
+    """
+    if node is None:
+        return True
+
+    ntype = node['type']
+    if ntype == 'true':
+        return True
+    if ntype == 'and':
+        return all(evaluate_scope(child, context) for child in node['nodes'])
+    if ntype == 'or':
+        return any(evaluate_scope(child, context) for child in node['nodes'])
+    if ntype == 'not':
+        return not evaluate_scope(node['node'], context)
+    if ntype == 'pctype':
+        return context.get('computertypeid') == node['id']
+    if ntype == 'app':
+        return node['id'] in (context.get('appids') or ())
+
+    # validate_scope should have caught this; be strict rather than silently
+    # matching an endpoint against an unknown node.
+    raise ScopeError(f'unknown scope node type at evaluation: {ntype!r}')