Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import

Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.

What arrives:

  endpoints plugin  - a ShopDB agent reporting endpoint state, telemetry,
                      services and interfaces, with a command channel the agent
                      long-polls. Ingest stays on the generic collector route
                      (ADR-006); the agent key is a UI-managed single-scope
                      token whose revocation is an instant kill switch.
  computers policies - Configuration and Business Policies (RMM-style), scope
                      trees validated at write time, actions checked against
                      the endpoints command allowlist. Feature-gated: every
                      route 404s while the setting is off, so enabling it can
                      never half-expose the surface.
  asset CSV import  - templates, a collector script carrying the site path, and
                      a validate-then-commit import. A plugin can own a virtual
                      column plus a post-write row hook, which is how a PC's
                      machine number becomes a controls link without a
                      hand-built relationships sheet.
  agent/            - the Windows agent and its installer.

Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.

MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).

Review fixes carried on top:

  - six policy GETs were jwt_required(optional=True), which authenticates a
    caller without checking anything; they require computers.view now, and six
    redundant jwt_required decorators are gone
  - the agent installer took a default BaseUrl and CollectorKey, so a mistyped
    run silently pointed an endpoint at the wrong instance; both are mandatory
    and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
  - a committing CSV import wrote no audit row

Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.

Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.

tests/test_import_sources.py

@@ -0,0 +1,201 @@
+"""Foreign import-source adapters: transforming Azure and Thor exports.
+
+These adapters are pure - they turn a foreign CSV into the canonical per-table
+CSVs the import engine already understands, and touch no database. The tests
+check the transform output directly, so they run without app context.
+"""
+import csv
+import io
+
+import pytest
+
+from shopdb.core.services.importsources import (
+    AzureDeviceGroupSource, ImportSourceError, ThorSource, detect_source,
+    get_source,
+)
+
+
+def _rows(text):
+    return list(csv.DictReader(io.StringIO(text)))
+
+
+AZURE_CSV = (
+    'displayName,objectType,mail,userType,id,deviceId,userPrincipalName,isUser,isGroup,isGuest\n'
+    'F13HSGH4,device,,,11111111-1111-1111-1111-111111111111,aaaa1111-bbbb-2222-cccc-333333333333,,False,False,False\n'
+    'E3P5M0B4,device,,,22222222-2222-2222-2222-222222222222,dddd4444-eeee-5555-ffff-666666666666,,False,False,False\n'
+)
+
+
+def test_azure_matches_signature():
+    header = ['displayName', 'objectType', 'deviceId', 'id']
+    assert AzureDeviceGroupSource.matches(header)
+    assert detect_source(header) is AzureDeviceGroupSource
+
+
+def test_azure_transform_builds_assets_and_computers():
+    produced = AzureDeviceGroupSource.transform(AZURE_CSV)
+    assert set(produced) == {'assets', 'computers'}
+
+    assets = _rows(produced['assets'])
+    assert [a['assetnumber'] for a in assets] == ['F13HSGH4', 'E3P5M0B4']
+    assert all(a['assettypeid'] == 'computer' for a in assets)
+    assert all(a['statusid'] == 'In Use' for a in assets)
+
+    computers = _rows(produced['computers'])
+    assert computers[0]['assetid'] == 'F13HSGH4'
+    assert computers[0]['hostname'] == 'F13HSGH4'
+    assert computers[0]['azuredeviceid'] == 'aaaa1111-bbbb-2222-cccc-333333333333'
+    assert 'computertypeid' not in computers[0]
+
+
+def test_azure_transform_stamps_computertype():
+    produced = AzureDeviceGroupSource.transform(
+        AZURE_CSV, options={'computertype': 'Shopfloor PC'})
+    computers = _rows(produced['computers'])
+    assert all(c['computertypeid'] == 'Shopfloor PC' for c in computers)
+
+
+def test_azure_transform_carries_objectid():
+    produced = AzureDeviceGroupSource.transform(AZURE_CSV)
+    computers = _rows(produced['computers'])
+    # The CSV 'id' column is the Entra directory object id.
+    assert computers[0]['azureobjectid'] == '11111111-1111-1111-1111-111111111111'
+
+
+def test_azure_transform_stamps_category():
+    produced = AzureDeviceGroupSource.transform(
+        AZURE_CSV, options={'category': 'SFLD_DYN_Device_Group_X'})
+    computers = _rows(produced['computers'])
+    assert all(c['azuredevicecategory'] == 'SFLD_DYN_Device_Group_X'
+               for c in computers)
+
+
+def test_azure_aligns_matched_hostname_to_existing_asset():
+    # A case-insensitive hostname match routes the device to its real asset and
+    # emits NO new asset row, so the import updates in place instead of creating
+    # a duplicate. Unmatched devices still create a new asset keyed on hostname.
+    produced = AzureDeviceGroupSource.transform(
+        AZURE_CSV, options={'hostname_map': {'f13hsgh4': '13NSGH4'}})
+
+    computers = _rows(produced['computers'])
+    by_host = {c['hostname']: c for c in computers}
+    # Matched device points at the existing assetnumber, not its hostname.
+    assert by_host['F13HSGH4']['assetid'] == '13NSGH4'
+    # Unmatched device falls back to its hostname (new asset).
+    assert by_host['E3P5M0B4']['assetid'] == 'E3P5M0B4'
+
+    # Only the unmatched device produces an asset row.
+    assets = _rows(produced['assets'])
+    assert [a['assetnumber'] for a in assets] == ['E3P5M0B4']
+
+
+def test_azure_all_matched_emits_no_assets_file():
+    produced = AzureDeviceGroupSource.transform(
+        AZURE_CSV,
+        options={'hostname_map': {'f13hsgh4': '13NSGH4', 'e3p5m0b4': 'E9Q'}})
+    # Every device matched, so there is nothing to create and no assets file.
+    assert 'assets' not in produced
+    computers = _rows(produced['computers'])
+    assert {c['assetid'] for c in computers} == {'13NSGH4', 'E9Q'}
+
+
+def test_azure_skips_non_device_rows_and_dedupes():
+    text = (
+        'displayName,objectType,mail,userType,id,deviceId,userPrincipalName,isUser,isGroup,isGuest\n'
+        'F13HSGH4,device,,,1,aaaa,,False,False,False\n'
+        'F13HSGH4,device,,,1,aaaa,,False,False,False\n'
+        'Some User,user,u@x,,2,,u@x,True,False,False\n'
+    )
+    computers = _rows(AzureDeviceGroupSource.transform(text)['computers'])
+    assert [c['hostname'] for c in computers] == ['F13HSGH4']
+
+
+def test_azure_rejects_wrong_header():
+    with pytest.raises(ImportSourceError):
+        AzureDeviceGroupSource.transform('a,b,c\n1,2,3\n')
+
+
+THOR_CSV = (
+    '"Host Name","IP Address","LOGON","OS Version","OS ID","Last Updated","Boot Time","Location","Model","BIOS Manf."\n'
+    '"F67HSGH4","10.0.0.5","F67HSGH4\\ShopFloor","10.0.26100","24H2","2026-01-02 03:04:05","2026-01-01 06:00:00","S13","Dell Pro Tower Plus QBT1250","Dell Inc."\n'
+)
+
+
+def test_thor_matches_signature():
+    header = ['Host Name', 'BIOS Manf.', 'OS Version', 'Model']
+    assert ThorSource.matches(header)
+    assert detect_source(header) is ThorSource
+
+
+def test_thor_transform_builds_lookup_and_subtype_files():
+    produced = ThorSource.transform(THOR_CSV)
+    assert 'vendors' in produced
+    assert 'models' in produced
+    assert 'operatingsystems' in produced
+    assert 'assets' in produced
+    assert 'computers' in produced
+
+    assert _rows(produced['vendors'])[0]['vendor'] == 'Dell Inc.'
+    assert _rows(produced['models'])[0]['modelnumber'] == 'Dell Pro Tower Plus QBT1250'
+    assert _rows(produced['operatingsystems'])[0]['osname'] == '10.0.26100 (24H2)'
+
+    computer = _rows(produced['computers'])[0]
+    assert computer['hostname'] == 'F67HSGH4'
+    assert computer['vendorid'] == 'Dell Inc.'
+    assert computer['modelnumberid'] == 'Dell Pro Tower Plus QBT1250'
+    assert computer['osid'] == '10.0.26100 (24H2)'
+    assert computer['loggedinuser'] == 'ShopFloor'
+    assert computer['lastreporteddate'] == '2026-01-02 03:04:05'
+    assert computer['lastboottime'] == '2026-01-01 06:00:00'
+
+    asset = _rows(produced['assets'])[0]
+    assert asset['locationid'] == 'S13'
+    assert asset['assettypeid'] == 'computer'
+    assert asset['statusid'] == 'In Use'
+
+
+THOR_TWO_CSV = (
+    '"Host Name","IP Address","LOGON","OS Version","OS ID","Last Updated","Boot Time","Location","Model","BIOS Manf."\n'
+    '"F67HSGH4","10.0.0.5","F67HSGH4\\ShopFloor","10.0.26100","24H2","2026-01-02 03:04:05","2026-01-01 06:00:00","S13","Dell Pro Tower Plus QBT1250","Dell Inc."\n'
+    '"FDKKSGH4","10.0.0.6","FDKKSGH4\\ShopFloor","10.0.26100","24H2","2026-01-03 03:04:05","2026-01-02 06:00:00","G8","Dell Pro Tower Plus QBT1250","Dell Inc."\n'
+)
+
+
+def test_thor_stamps_computertype():
+    computer = _rows(
+        ThorSource.transform(THOR_CSV, options={'computertype': 'Shop Floor PC'})['computers'])[0]
+    assert computer['computertypeid'] == 'Shop Floor PC'
+
+
+def test_thor_aligns_matched_hostname_to_existing_asset():
+    # A case-insensitive hostname match routes the host to its real asset and
+    # emits NO new asset row, so the import updates in place instead of creating
+    # a duplicate. Unmatched hosts still create a new asset keyed on hostname.
+    produced = ThorSource.transform(
+        THOR_TWO_CSV, options={'hostname_map': {'f67hsgh4': 'PC-1001'}})
+
+    computers = _rows(produced['computers'])
+    by_host = {c['hostname']: c for c in computers}
+    assert by_host['F67HSGH4']['assetid'] == 'PC-1001'
+    assert by_host['FDKKSGH4']['assetid'] == 'FDKKSGH4'
+
+    assets = _rows(produced['assets'])
+    assert [a['assetnumber'] for a in assets] == ['FDKKSGH4']
+
+
+def test_thor_all_matched_emits_no_assets_file():
+    produced = ThorSource.transform(
+        THOR_TWO_CSV,
+        options={'hostname_map': {'f67hsgh4': 'PC-1001', 'fdkksgh4': 'PC-1002'}})
+    assert 'assets' not in produced
+    computers = _rows(produced['computers'])
+    assert {c['assetid'] for c in computers} == {'PC-1001', 'PC-1002'}
+
+
+def test_get_source_unknown_raises():
+    with pytest.raises(ImportSourceError):
+        get_source('nope')
+
+
+def test_detect_source_none_for_unknown_header():
+    assert detect_source(['foo', 'bar']) is None