Clear the review blockers on the merged wilm work

A multi-agent review of the merged branch produced 29 findings that survived
adversarial verification. This clears the blockers and the four worth fixing
before it reaches main. The design held up well: path-traversal defence on all
four serving routes is clean, the agent's SHA-256 check fails closed, enqueue
authority is proportionate and audited, and no site literals or credential leaks
in the new C#.

THE TESTS NEVER RAN

pytest.ini sets testpaths = tests, so plugins/endpoints/tests/ was collected ZERO
times. Fifty-five lines of delivery assertions were dead code, and run directly
five of them failed. plugins/printedparts/tests/ had been invisible for longer.

That is the worst shape a gap can take: the suite is green, the coverage is
imaginary, and the collector auth on a download route could be deleted without
anything going red.

Both files moved under tests/. tests/test_no_hidden_tests.py now fails if a test
file appears anywhere pytest will not look; it caught a staged copy inside the
installer bundle on its first run, and was verified by planting a violation.

The five failures were stale expectations, and they split by cause. His: the
applications panel made six panels where his test said five, and 'applications'
belongs in KNOWN_PAYLOAD_KEYS now that it is persisted - his own test contradicted
his own feature. Ours: we renamed the events card to activity, moved the
fleet-wide cards to endpoints.fleet, and made the list route a bare route('')
which Flask rejects unless the blueprint is mounted under a prefix.

FOUR FIXES

The snapshot table's key was a 32-bit autoincrement on a table rewritten
wholesale every check-in, so it climbs at (endpoints x apps) per report and
exhausts on MySQL. Widened while the table is unreleased. Widening it broke every
insert on SQLite, which only autoincrements a column typed exactly INTEGER
PRIMARY KEY, so it carries a dialect variant - and nothing caught either fault,
because no test inserted into that table at all. One does now.

Ingest truncated displayname but not displayversion or publisher. On strict MySQL
an over-long registry value fails the whole check-in transaction, so one machine
with a malformed Uninstall entry goes silent entirely. SQLite does not enforce
lengths, so the suite would never have shown it.

update_app's version: branch pinned AppVersion.sha256, but that route resolves the
version to its APPLICATION and serves application.installpath - so the digest
described an artifact the route never sends, and where it was null the agent had
nothing to verify and failed open. It now pins the file actually served.

The same branch ended the delivery URL at the id, so an MSI arrived with no
extension and could not be routed through msiexec. It carries the filename now.

Agent packaging moved off require_role('admin') to a new endpoints.agentpackage
permission. The generated ZIP carries a live collector key, so this is a heavier
grant than deploying to one endpoint - and a role cannot be given to one person
for one job the way a permission can.

Full suite 2231 passed (2198 before, plus the 22 that never ran and 11 new),
frontend 314.

plugins/endpoints/api/adoption.py

@@ -23,7 +23,7 @@ import zipfile
 
 from flask import request, current_app, send_file
 
-from shopdb.api import error_response, ErrorCodes, require_role
+from shopdb.api import error_response, ErrorCodes, require_permission
 
 from .routes import endpoints_bp
 
@@ -66,7 +66,7 @@ def _build_readme():
 
 
 @endpoints_bp.route('/adoptiontool/package', methods=['POST'])
-@require_role('admin')
+@require_permission('endpoints.agentpackage')
 def package_adoption_tool():
     """Bundle the adoption tool as a downloadable ZIP.
 

plugins/endpoints/api/routes.py

@@ -1097,14 +1097,39 @@ def _resolve_update_app(target):
         if application is None or not application.installpath:
             return None, ('Application for appversionid {} has no installer package'
                           .format(ident))
+        # PIN THE FILE THAT IS ACTUALLY SERVED.
+        #
+        # /api/endpoints/apppackage/<appversionid> resolves the version to its
+        # APPLICATION and serves application.installpath - there is no
+        # version-specific artifact behind that route. So version.sha256 pinned
+        # a digest that need not describe the bytes delivered, and where it was
+        # null the descriptor carried no digest at all and the agent had nothing
+        # to verify against. Deriving it from the application, exactly as the
+        # app branch does, makes the pin describe the delivery.
+        sha256 = (application.installersha256
+                  or _backfill_app_installer_sha256(application))
+        if not sha256:
+            return None, ('Application for appversionid {} has an installer '
+                          'package with no recorded SHA-256, so the delivery '
+                          'cannot be verified'.format(ident))
+
+        # Carry the real filename so the agent's Path.GetExtension routes an MSI
+        # through msiexec. Without it every MSI push failed, because this branch
+        # ended the URL at the id and the extension was simply absent.
+        filename = _hosted_package_filename(application)
+        installpath = '/api/endpoints/apppackage/{}'.format(version.appversionid)
+        if filename:
+            installpath = '/api/endpoints/apppackage/app/{}/{}'.format(
+                application.appid, filename)
+
         return {
             'source': 'version',
             'appversionid': version.appversionid,
             'appid': version.appid,
             'appname': application.appname,
             'version': version.version,
-            'installpath': '/api/endpoints/apppackage/{}'.format(version.appversionid),
-            'sha256': version.sha256,
+            'installpath': installpath,
+            'sha256': sha256,
             'silentargs': version.silentargs,
         }, None
 

plugins/endpoints/api/siteagent.py

@@ -31,7 +31,8 @@ import zipfile
 
 from flask import request, current_app, send_file
 
-from shopdb.api import success_response, error_response, ErrorCodes, require_role
+from shopdb.api import (success_response, error_response, ErrorCodes,
+                        require_permission)
 
 from .routes import endpoints_bp
 
@@ -335,14 +336,14 @@ def _siteagent_status_payload():
 
 
 @endpoints_bp.route('/siteagent', methods=['GET'])
-@require_role('admin')
+@require_permission('endpoints.agentpackage')
 def get_siteagent():
     """Report whether the site agent build has a prebuilt binary staged."""
     return success_response(_siteagent_status_payload())
 
 
 @endpoints_bp.route('/siteagent/package', methods=['POST'])
-@require_role('admin')
+@require_permission('endpoints.agentpackage')
 def package_siteagent():
     """Bundle the site agent as a downloadable ZIP.
 

plugins/endpoints/migrations/versions/0004_endpoint_applications.py

@@ -38,7 +38,17 @@ def upgrade():
 
     op.create_table(
         'endpointapplications',
-        sa.Column('endpointapplicationid', sa.Integer(), nullable=False),
+        # BigInteger, not Integer. This is a SNAPSHOT table: every check-in
+        # deletes the endpoint's rows and reinserts them, so the surrogate key
+        # burns through values at (endpoints x installed apps) per report and
+        # never reuses them. A 32-bit key exhausts on MySQL - a few hundred
+        # endpoints reporting a few hundred apps every 15 minutes reaches 2.1
+        # billion inside a couple of years, and the failure is an insert that
+        # stops working forever. SQLite has no such ceiling, so no test would
+        # have caught it. Changed while the table is still unreleased.
+        sa.Column('endpointapplicationid',
+                  sa.BigInteger().with_variant(sa.Integer, 'sqlite'),
+                  nullable=False),
         sa.Column('assetid', sa.Integer(), nullable=False),
         sa.Column('displayname', sa.String(length=255), nullable=False),
         sa.Column('displayversion', sa.String(length=128), nullable=True),

plugins/endpoints/models/endpoint.py

@@ -282,7 +282,16 @@ class EndpointApplication(db.Model):
 
     __tablename__ = 'endpointapplications'
 
-    endpointapplicationid = db.Column(db.Integer, primary_key=True)
+    # BigInteger on MySQL, Integer on SQLite. This table is rewritten wholesale
+    # on every check-in, so the key never stops climbing and a 32-bit column
+    # eventually stops accepting inserts - see the migration for the arithmetic.
+    #
+    # The variant is not decoration: SQLite only autoincrements a column typed
+    # exactly INTEGER PRIMARY KEY, so a plain BigInteger makes every insert fail
+    # with a NOT NULL violation on the test database. Nothing caught that,
+    # because no test inserts into this table.
+    endpointapplicationid = db.Column(
+        db.BigInteger().with_variant(db.Integer, 'sqlite'), primary_key=True)
 
     # The asset this install belongs to. Not unique: many applications per
     # endpoint. CASCADE so removing the asset clears its SBOM snapshot.

plugins/endpoints/plugin.py

@@ -78,6 +78,20 @@ def _as_json(value):
     return json.dumps(value, separators=(',', ':'), sort_keys=True)
 
 
+def _clamp(value, length):
+    """A string bounded to its column width, or None.
+
+    Ingest writes whatever an endpoint reports. A registry value longer than its
+    column is not a data-quality nuisance on MySQL: in strict mode it raises
+    Data too long and fails the whole check-in transaction, so one machine with
+    a malformed Uninstall entry stops reporting telemetry and events entirely.
+    """
+    if value is None:
+        return None
+    text = str(value).strip()
+    return text[:length] or None
+
+
 class EndpointsPlugin(BasePlugin):
     """Endpoint RMM telemetry ingest plus read/dashboard surface."""
 
@@ -153,6 +167,17 @@ class EndpointsPlugin(BasePlugin):
             # heavier grant than a routine service/process toggle.
             ('endpoints.deploy',
              'Run staged scripts and push application installers to endpoints', 'endpoints'),
+            # Packaging the agent is a heavier grant than deploying to one, and
+            # it is not the same job either: the generated ZIP carries a live
+            # collector key stamped into its install script, so whoever holds
+            # this can mint a distributable artefact containing the fleet-wide
+            # command-channel credential. It was require_role('admin'), which
+            # this codebase otherwise reserves for the few genuinely
+            # role-shaped checks - and a role cannot be granted to one person
+            # for one job the way a permission can.
+            ('endpoints.agentpackage',
+             'Generate and download an agent or adoption-tool package '
+             '(contains a live collector key)', 'endpoints'),
             # Collector service scope for the endpoint agent. A managed token
             # scoped to ONLY this authorizes the endpoints collector and nothing
             # else, so rotating/revoking it stops endpoint-agent traffic without
@@ -641,11 +666,18 @@ class EndpointsPlugin(BasePlugin):
             if not displayname:
                 warnings.append('skipped application with no displayname')
                 continue
+            # TRUNCATE EVERY STRING, not just the name. On MySQL in strict
+            # mode an over-long value raises Data too long and takes down the
+            # WHOLE check-in transaction, so one machine with a malformed
+            # Uninstall registry entry stops reporting entirely - telemetry,
+            # events and all - until somebody uninstalls the offender. SQLite
+            # does not enforce declared lengths, so no test here would ever see
+            # it. displayname was already clamped; these two were not.
             db.session.add(EndpointApplication(
                 assetid=assetid,
                 displayname=displayname[:255],
-                displayversion=(item.get('displayversion') or None),
-                publisher=(item.get('publisher') or None),
+                displayversion=_clamp(item.get('displayversion'), 128),
+                publisher=_clamp(item.get('publisher'), 255),
             ))
             stored += 1
         return stored

plugins/endpoints/tests/test_plugin.py

@@ -1,226 +0,0 @@
-"""Endpoints plugin smoke + collector-contract tests.
-
-Asserts the plugin loads cleanly, satisfies the framework contract, and that
-its ADR-006 collector schema is well formed. Extend with DB-backed ingest tests
-as the plugin grows.
-"""
-
-from flask import Blueprint
-
-from plugins.endpoints.plugin import EndpointsPlugin, CURATED_EVENT_IDS
-from plugins.endpoints.eventcatalog import EVENT_CATALOG, CRITICAL_EVENT_IDS
-from plugins.endpoints.commands import (
-    COMMAND_ALLOWLIST, is_allowed_action, elevated_permission_for,
-    requires_target,
-)
-
-
-def test_endpoints_plugin_meta_is_valid():
-    """meta returns a PluginMeta with the expected name and prefix."""
-    plugin = EndpointsPlugin()
-    assert plugin.meta.name == 'endpoints'
-    assert plugin.meta.api_prefix == '/api/endpoints'
-    assert 'computers' in plugin.meta.dependencies
-
-
-def test_endpoints_plugin_get_blueprint_returns_blueprint():
-    """get_blueprint returns a Flask Blueprint, not None."""
-    plugin = EndpointsPlugin()
-    assert isinstance(plugin.get_blueprint(), Blueprint)
-
-
-def test_endpoints_plugin_get_models_returns_models():
-    """get_models returns the two telemetry tables."""
-    plugin = EndpointsPlugin()
-    models = plugin.get_models()
-    assert len(models) >= 2
-    for model in models:
-        assert hasattr(model, '__tablename__')
-
-
-def test_endpoints_collector_schema_identity_is_hostname():
-    """ADR-006 schema declares hostname as the identity field."""
-    plugin = EndpointsPlugin()
-    schema = plugin.get_collector_schema()
-    assert schema is not None
-    assert schema['identityfield'] == 'hostname'
-    assert 'hostname' in schema['fields']['required']
-
-
-def test_endpoints_curated_event_ids_include_core_security_ids():
-    """The curated catalog covers the agreed logon/stability IDs."""
-    for eventid in (4624, 4625, 4634, 4672, 4697, 4698, 41, 4618, 55, 7011,
-                    10016, 1001):
-        assert eventid in CURATED_EVENT_IDS
-
-
-def test_curated_ids_match_the_event_catalog():
-    """Ingest filter and display catalog cannot drift apart."""
-    assert CURATED_EVENT_IDS == frozenset(EVENT_CATALOG)
-
-
-def test_critical_event_ids_are_a_subset_of_the_catalog():
-    """Dashboard stability card only queries known critical IDs."""
-    assert set(CRITICAL_EVENT_IDS)
-    assert set(CRITICAL_EVENT_IDS).issubset(set(EVENT_CATALOG))
-
-
-def test_endpoints_asset_panels_are_computer_scoped():
-    """Every asset panel targets computer assets only, with an endpoint."""
-    plugin = EndpointsPlugin()
-    panels = plugin.get_asset_panels()
-    assert len(panels) == 5
-    for panel in panels:
-        assert panel['assettypes'] == ['computer']
-        assert panel['endpoint'].startswith('/api/endpoints/asset/{assetid}')
-        assert panel['render'] in ('keyvalue', 'list', 'table')
-
-
-def test_services_and_processes_are_split_into_two_filtered_cards():
-    """The single services card is now two cards filtered by row kind, both
-    reading the same services snapshot so a future top-20 collector is just
-    another filtered panel over the same feed."""
-    plugin = EndpointsPlugin()
-    panels = {p['id']: p for p in plugin.get_asset_panels()}
-    services = panels['endpoints-tracked-services']
-    processes = panels['endpoints-tracked-processes']
-    assert services['title'] == 'Tracked Services'
-    assert processes['title'] == 'Tracked Processes'
-    assert services['filter'] == {'key': 'kind', 'equals': 'service'}
-    assert processes['filter'] == {'key': 'kind', 'equals': 'process'}
-    # Both slice one endpoint feed.
-    assert services['endpoint'] == processes['endpoint']
-    assert services['endpoint'].endswith('/services')
-    # Command wiring carries over to both cards.
-    assert services['permission'] == 'endpoints.command'
-    assert processes['permission'] == 'endpoints.command'
-
-
-def test_recent_events_card_is_last_and_starts_collapsed():
-    """Events is the lowest-priority card: forced to the bottom of the column
-    and collapsed by default."""
-    plugin = EndpointsPlugin()
-    panels = plugin.get_asset_panels()
-    events = next(p for p in panels if p['id'] == 'endpoints-events')
-    assert events['collapsedDefault'] is True
-    assert events['position'] == max(p['position'] for p in panels)
-
-
-def test_endpoints_dashboard_widgets_declared():
-    """Quiet + stability exception cards are declared and hide when empty."""
-    plugin = EndpointsPlugin()
-    widgets = plugin.get_dashboard_widgets()
-    ids = {w['id'] for w in widgets}
-    assert {'endpoints-quiet', 'endpoints-stability'} <= ids
-    for widget in widgets:
-        assert widget['render'] == 'exceptions'
-        assert widget['empty'] == 'hide'
-        assert widget['permission'] == 'endpoints.view'
-
-
-def test_endpoints_navigation_item_points_at_fleet_view():
-    """The fleet list view is exposed as a single /endpoints nav item."""
-    plugin = EndpointsPlugin()
-    items = plugin.get_navigation_items()
-    routes = {item['route'] for item in items}
-    assert '/endpoints' in routes
-
-
-def test_system_restart_is_allowlisted_and_policy_selectable():
-    """Restart is a real allowlisted action, so the policy builder (which
-    reads COMMAND_ALLOWLIST) surfaces it as a selectable action."""
-    assert is_allowed_action('system_restart')
-    assert 'system_restart' in COMMAND_ALLOWLIST
-    assert COMMAND_ALLOWLIST['system_restart']['label'] == 'Restart computer'
-
-
-def test_system_restart_requires_the_elevated_restart_permission():
-    """Restart maps to endpoints.restart so the enqueue route can demand a
-    heavier grant than the routine endpoints.command toggle. Routine actions
-    carry no elevated requirement."""
-    assert elevated_permission_for('system_restart') == 'endpoints.restart'
-    assert elevated_permission_for('mtconnect_start') is None
-
-
-def test_endpoints_declares_the_restart_permission():
-    """The dedicated restart permission is declared for role-grid seeding."""
-    plugin = EndpointsPlugin()
-    permissions = {name for name, *_ in plugin.get_permissions()}
-    assert 'endpoints.restart' in permissions
-    assert 'endpoints.command' in permissions
-
-
-def test_collect_now_is_allowlisted_targetless_and_routine():
-    """collect_now forces an off-cadence telemetry report. It is allowlisted so
-    the enqueue route accepts it, carries no wire target (fixed action), and
-    needs no elevated grant beyond endpoints.command."""
-    assert is_allowed_action('collect_now')
-    assert 'collect_now' in COMMAND_ALLOWLIST
-    assert not requires_target('collect_now')
-    assert elevated_permission_for('collect_now') is None
-
-
-def test_run_script_and_update_app_are_allowlisted_delivery_actions():
-    """The two software-delivery actions are allowlisted with a targetkind so
-    the enqueue route accepts them and carries their subject on the wire:
-    run_script carries a script name, update_app carries an appversionid."""
-    for action in ('run_script', 'update_app'):
-        assert is_allowed_action(action)
-        assert action in COMMAND_ALLOWLIST
-        assert requires_target(action)
-    assert COMMAND_ALLOWLIST['run_script']['targetkind'] == 'script'
-    assert COMMAND_ALLOWLIST['update_app']['targetkind'] == 'app'
-
-
-def test_delivery_actions_require_the_elevated_deploy_permission():
-    """run_script and update_app both demand endpoints.deploy ON TOP OF
-    endpoints.command, so pushing software is a strictly heavier grant than a
-    routine service/process toggle."""
-    assert elevated_permission_for('run_script') == 'endpoints.deploy'
-    assert elevated_permission_for('update_app') == 'endpoints.deploy'
-
-
-def test_endpoints_declares_the_deploy_permission():
-    """The dedicated software-delivery permission is declared for role seeding."""
-    plugin = EndpointsPlugin()
-    permissions = {name for name, *_ in plugin.get_permissions()}
-    assert 'endpoints.deploy' in permissions
-
-
-def test_known_payload_keys_cover_persisted_and_relayed_fields():
-    """The ingest allow-set must list every key the handler consumes today, so
-    those keys are never mislogged as 'unpersisted'. Anything the agent sends
-    outside this set is logged for observability before columns exist."""
-    from plugins.endpoints.plugin import KNOWN_PAYLOAD_KEYS
-
-    persisted_or_relayed = {
-        'hostname', 'adopt', 'loggedinuser', 'serialnumber', 'pctype',
-        'machinenumber', 'vendorname', 'modelnumber', 'osname', 'ipaddress',
-        'agentversion', 'lastboottime', 'uptimeseconds', 'idleseconds',
-        'cpupercent', 'memorypercent', 'services', 'ports', 'interfaces',
-        'scheduledtasks', 'events',
-    }
-    assert persisted_or_relayed <= KNOWN_PAYLOAD_KEYS
-
-
-def test_new_hardware_telemetry_keys_are_logged_not_silently_known():
-    """Disk/bios/memory inventory is collected at the agent before the server
-    persists it. Those keys must be OUTSIDE the known set so ingest logs them
-    rather than dropping them silently."""
-    from plugins.endpoints.plugin import KNOWN_PAYLOAD_KEYS
-
-    for key in ('disks', 'bios', 'memory', 'processor', 'osinfo', 'domain',
-                'applications'):
-        assert key not in KNOWN_PAYLOAD_KEYS
-
-
-
-def test_telemetrymeta_route_is_registered_for_freshness_gating():
-    """The WebUI gates panel refreshes on lastreportedat advancing, so the
-    lightweight freshness probe must be exposed on the plugin blueprint."""
-    from flask import Flask
-
-    application = Flask(__name__)
-    application.register_blueprint(EndpointsPlugin().get_blueprint())
-    rules = {rule.rule for rule in application.url_map.iter_rules()}
-    assert any(rule.endswith('/asset/<int:asset_id>/telemetrymeta') for rule in rules)

plugins/printedparts/tests/test_plugin.py

@@ -1,30 +0,0 @@
-"""Printedparts plugin smoke tests.
-
-Asserts the plugin loads cleanly and satisfies the framework contract.
-Replace and extend with domain tests as you build the plugin out.
-"""
-
-from plugins.printedparts.plugin import PrintedpartsPlugin
-
-
-def test_printedparts_plugin_meta_is_valid():
-    """PrintedpartsPlugin.meta returns a PluginMeta with the expected name."""
-    plugin = PrintedpartsPlugin()
-    assert plugin.meta.name == 'printedparts'
-    assert plugin.meta.api_prefix == '/api/printedparts'
-
-
-def test_printedparts_plugin_get_blueprint_returns_blueprint():
-    """get_blueprint returns a Flask Blueprint, not None."""
-    from flask import Blueprint
-    plugin = PrintedpartsPlugin()
-    assert isinstance(plugin.get_blueprint(), Blueprint)
-
-
-def test_printedparts_plugin_get_models_returns_a_model():
-    """get_models returns a list with at least one SQLAlchemy model."""
-    plugin = PrintedpartsPlugin()
-    models = plugin.get_models()
-    assert len(models) >= 1
-    for model in models:
-        assert hasattr(model, '__tablename__')