Files
shopdb-flask/docs/wiki-drafts/Architectural-Findings-2026-09-11.md
cproudlock 69a7997dc4 Build the plugin set a real site runs, and write down what shipped
The lean-build job proved the delete-a-plugin guarantee with two plugins and
four hand-picked marker components. It now builds a real twelve-plugin set, and
the assertions follow a rule instead of a hunch: one component from EVERY
omitted plugin must be absent, one from EVERY chosen plugin must be present, and
the backend tree is checked both ways. An omitted set that grows without its
check growing is how a leak ships.

CHANGELOG had not been touched across the whole range, so the largest
security-behaviour change in the product - per-PC endpoint credentials, the
shared-key refusal for enrolled PCs, server pinning, agent 1.20.1 and tray 1.7.1
- had no release record. An [Unreleased] section now carries it, written from
the commits rather than from memory.

CLAUDE.md and the agent quick-reference stated that ADR-016 bans storing
credentials in the registry. It does not, and this fleet depends on HKLM for its
collector key and server pins: the next person following that text removes them
and silently breaks reporting and pinning on every un-enrolled PC. Corrected,
along with two other statements that were false on this branch - the database is
MySQL, not PostgreSQL, and column names are lowercase concatenated, not
snake_case, which is the opposite of what the gate enforces.

The wiki drafts are updated against the code as it stands, each old finding
given a verdict with a file and line rather than left as an assertion from a
week ago, and a new page records the ADR conformance result - including that the
ADR set itself is not on this branch, so nothing here enforces it except the
naming and style script.

They are also scrubbed of the things that must not reach a published wiki: the
site name, the internal git server, developer home paths and the assistant
tooling. They now pass the repository's own publishability patterns, which is
what the scrub is for.
2026-09-18 13:31:25 -04:00

20 KiB

Architectural Findings - September 11, 2026

Scope: Full codebase analysis via Gortex semantic code intelligence Original severity claim: 22 findings (6 critical, 6 high, 10 medium) Status: RE-CHECKED against branch gh-wj at 35d4fcd41 on 2026-09-18. Most of the original findings do not survive a read of the code. This page keeps them, with the verdict and the evidence, so nobody acts on a claim that was never true.

How to read this page (added 2026-09-18)

The September 11 analysis was written from semantic queries, not from reading the files it names. Eight of its claims cite code that does not exist on this branch: a machines plugin that imports computers, a migration that spells a column last_reported_date, a ServiceNow plugin, MACHINE_TYPES = ['Lathe', 'Mill', 'CNC'], a hardcoded ['Bldg 300', 'Bldg 400'], Vuex, an absent CSP header, an unhandled requests.post. Each of those was grepped for on 2026-09-18 and returns nothing.

Every finding below now carries one of four verdicts:

  • REFUTED - the cited code does not exist, or the opposite is in the tree.
  • STALE - it was true once; the tree has since fixed it.
  • HOLDS (narrowed) - something real is there, but smaller or different than the original text claims.
  • UNVERIFIED - cannot be settled from this repository.

Effort and priority numbers in the original text were estimates with no stated basis. They are left where they sit for findings that still hold and struck where the finding does not.

Critical Findings (P0)

1. Circular Dependencies Between Plugins

Verdict: REFUTED as stated. HOLDS (narrowed) as a deferred-import coupling.

Original claim: computers imports machines and machines imports computers; import errors at runtime if plugins load in the wrong order; 25+ violations.

What the code shows (2026-09-18):

  • grep -rn "^from plugins\." plugins/ --include=*.py returns nothing. There is not one module-level cross-plugin import in the tree, so there is no load-order failure mode and no import cycle at import time.
  • grep -rn "from plugins.computers" plugins/machines/ returns nothing. The machines plugin does not reference computers at all, so the cycle the finding is built on does not exist.
  • 52 cross-plugin imports do exist, and every one of them is inside a function body, resolved on call. plugins/computers/plugin.py:414 and :1166 reach plugins.machines.models that way; plugins/endpoints/api/routes.py:235 and fourteen more reach plugins.computers.models.

What is actually there: a mutual reference pair between computers and endpoints, both deferred:

  • plugins/computers/api/policy_routes.py:44 imports plugins.endpoints.commands.elevated_permission_for, unguarded.
  • plugins/endpoints/commands.py:477 imports plugins.computers.services.policyresolver.effective_policy_for, wrapped in try/except that falls back to the base allowlist (commands.py:475-481).

So one side of that pair degrades when its partner is absent and the other raises. The same asymmetry appears at plugins/endpoints/api/agentauth.py:118, where resolve_hostname imports plugins.computers.models.Computer with no guard: an endpoints-without- computers site fails there rather than returning "unknown hostname".

What to do instead of the original remediation: nothing about load order. The open question is narrower and worth one pass: decide whether a deferred cross-plugin import must be guarded, and guard the unguarded ones. A linter rule banning plugins.* imports outright would break the working policy and enrollment paths listed above, which are the product's actual design.

2. Undeclared Import Dependencies

Verdict: REFUTED.

Original claim: 18 plugins import shopdb.core.models, 12 import shopdb.core.utils, 7 import shopdb.core.services.

What the code shows: grep -rn "from shopdb\.\(core\|utils\|extensions\|services\)" plugins/ --include=*.py returns 0 hits. Not one plugin deep-imports core. The only shopdb.* import in plugins/ outside shopdb.api is shopdb.plugins.alembic_template from migration env.py files, which is the shared Alembic runner ADR-008 sanctions.

ADR-001's rule that shopdb.api is the only core surface a plugin may import is being kept. The thing that is NOT enforced is raw SQL against core tables: see finding 3a below.

2a. Raw SQL against core tables from a plugin (added 2026-09-18)

Verdict: HOLDS (new, narrow).

plugins/computers/plugin.py:2080-2091 runs db.session.execute(db.text(...)) joining core assets, assetrelationships and assettypes inside the flask computers retype CLI command. Asset, AssetType, AssetRelationship and RelationshipType are all exported from shopdb.api (shopdb/api/__init__.py:47-48, :424, :438-439), so no missing export excuses it, and plugins/endpoints/api/routes.py:3981-3986 is the same pattern already corrected for BusinessUnit. The contract note at shopdb/__init__.py:135-142 names the reason: a raw query is coupling to the core schema that no contract bump can catch.

Blast radius: one admin CLI command, no serving route. Import rule not literally broken. Worth fixing because it is the kind of thing that gets copied.

3. Schema Drift: Models vs Migrations

Verdict: REFUTED as stated. HOLDS (narrowed) to one column.

Original claim: Computer.lastreporteddate in the model but last_reported_date in the migration; Printer.driver with no migration; AssetRelationship.label in a migration but not the model; 14 tables affected.

What the code shows:

  • grep -rn "last_reported_date" --include=*.py . returns nothing. The model spells it lastreporteddate (plugins/computers/models/computer.py:85) and so does every migration.
  • grep -rPn "sa\.Column\(\s*'[a-z]+_[a-z_]+'" plugins/*/migrations/versions/*.py migrations/versions/*.py returns nothing. There is no snake_case column name in any migration in the tree.
  • There is no shopdb/models/ directory to hold the claimed AssetRelationship.label; core models live under shopdb/core/.

The one real instance: plugins/notifications/models/notification.py:158 declares alertscope = db.Column(db.JSON, nullable=True) while the migration that creates it, plugins/notifications/migrations/versions/0006_add_fleet_alerts.py:45, creates sa.Column('alertscope', sa.Text, nullable=True) and its own docstring at line 8 calls it "the alertscope TEXT field (JSON string)". Commit f484be8a3 fixed the two neighbouring columns to agree with 0006 and left this one. Nothing breaks on MySQL 8 - the SQLAlchemy JSON type serialises into TEXT and reads back

  • and consumers already normalise both shapes (plugins/endpoints/api/routes.py:4011-4020). The cost is that the model is not a true description of the deployed column, which is what autogenerate reads.

Related gap that is real: the naming gate cannot see migrations at all. scripts/check-naming-and-style.sh:236 greps ^\s+[a-z]+_[a-z_]+\s*=\s*db\.Column, which only matches a model attribute assignment; migrations write sa.Column('name', ...) with the name as a positional string. That is how the shipped build of plugins/endpoints/migrations/versions/0021_client_certificates.py created client_cert_fingerprint, cert_issued_at and cert_expires_at and passed the gate (git show 5cd533239:plugins/endpoints/migrations/versions/0021_client_certificates.py, lines 20-28). The tree is conformant today, because 0021 and 0022 are now no-ops and 0023 renamed the survivor; the gate simply would not notice the next one.

4. Single Points of Failure

Verdict: UNVERIFIED from this repository, and partly STALE.

Replication, load balancing and orchestration are properties of the reference site deployment, not of this tree, so nothing here can confirm or deny them. Two parts of the original text are wrong about what the repo ships:

  • Restart policy is configured, not absent: docker-compose.yml:24, :104 and :120 set restart: unless-stopped, and three services carry healthcheck: blocks (:38, :112, :219).
  • Backups are no longer "manual mysqldump weekly, stored on the same server". See finding 4a.

The underlying point - one database, one app host, air gap limits the options - is a statement about the site, and belongs in a deployment document rather than a code findings page. The 200+ hour estimate has no basis in anything measured.

4a. Backups (added 2026-09-18, correcting the original text)

Verdict: STALE. The repo ships a backup system.

scripts/backup/ contains backup.sh, restore.sh, status.sh, watch.sh, simulate-restore.sh and test-restore-datarev.sh. Retention is enforced by restic (scripts/backup/backup.sh:134-141: --keep-daily 5 --keep-weekly 4 --keep-monthly 6 --prune), destinations include SMB and S3 as well as local (scripts/backup/README.md), each set carries manifest.json with schema revisions for compatibility checking, and restore runs from the Settings > Backups UI with a schema compatibility check. A restore-simulation harness is in the tree.

5. Plugin Migration Chain Validation Gaps

Verdict: STALE as written, and the credited fix is the wrong artifact.

Validation is in the test suite, in this repository, and has been for longer than the /migration-check skill: tests/test_plugin_migrations.py holds nine tests covering owners-versus-models, chain shape, no-op anchors and idempotency, and pins each chain's head in EXPECTED_HEAD_REVISION (tests/test_plugin_migrations.py:125). tests/test_plugins/test_endpoints_credentials_migration.py runs the 0020-0023 chain twice from both a legacy and a fresh schema, plus downgrade and re-upgrade. The endpoints chain today is 24 revisions with one root and one head and no dangling down_revision.

The branch example in the original text (003_add_policies.py with down_revision=001) is invented; no such file exists under plugins/computers/migrations/versions/, which holds 6 revisions.

The real gap, which the original text missed: no test asserts that a plugin chain touches only the tables it owns. plugins/endpoints/migrations/versions/0023_endpoint_credentials.py declares plugin_name = 'endpoints' (line 33) and then runs op.drop_index(..., table_name='assets') at line 73 and op.drop_column('assets', column) at line 76 against the core assets table, cleaning up after the shipped 0021 which had added those columns from the same plugin chain. assets is not, and cannot be, in PLUGIN_TABLE_OWNERS['endpoints'] (shopdb/plugins/alembic_template.py:57-63), so the metadata ownership filter never saw it. Every step is guarded on the live schema and the forward direction is clean, so nothing is broken; nothing stops the next one either.

6. Implicit Database Connection Sharing

Verdict: REFUTED as a blanket claim.

Savepoints are in use where concurrency actually bites: plugins/endpoints/plugin.py:1318, :1387 and :1692 all wrap work in with db.session.begin_nested():, and the application-delta path reconciles a unique-constraint collision inside that savepoint rather than sinking the check-in. The original text's example - a printers rollback silently discarding a computers write in the same request - is not backed by any cited call site, and none was found.

A per-plugin transaction policy is still not written down anywhere. That is a documentation item for PLUGIN-DEVELOPMENT.md, not a critical data-integrity finding.

High Priority Findings (P1)

7. Frontend Plugin Import Violations

Verdict: STALE. Enforced and passing.

scripts/check-naming-and-style.sh runs "Checking for cross-plugin / escaping imports in plugin frontends" and the whole script exits 0 on this branch. The established pattern is a core component consumed through the @/ alias: 97 plugin frontend files already import that way, and the kiosk allowlist editor added in this range follows it (frontend/src/components/NetworkAllowlistPicker.vue, consumed at plugins/usb/frontend/views/USBSettings.vue:108 and plugins/printedparts/frontend/views/PrintedPartsSettings.vue:164).

8. Hardcoded Site Literals in Plugins

Verdict: REFUTED as stated. HOLDS (narrowed) as a gate coverage gap.

grep -rn "MACHINE_TYPES" . --include=*.py and grep -rn "Bldg 300\|Bldg 400" . --include=*.py both return nothing. The ADR-015 block in scripts/check-naming-and-style.sh:392-462 runs and passes, and a control pattern over the same directories returns 1995 lines, so the pipeline is not vacuous.

The narrowed finding: the ADR-015 scan is scoped at scripts/check-naming-and-style.sh:436-440 to plugins/ shopdb/ scripts/ deploy/ frontend/src/ tools/ with --include on .py/.vue/.js/.ps1/.psm1/.sh/.iss/.json/.html. agent/ is not in that list and .cs is not an included extension, so roughly 2,500 lines of C# added in this range - AgentCredentials.cs, AgentHttpFactory.cs, EnrollmentService.cs, AgentPipeServer.cs, ProtectedFileSystem.cs, StrictModePreamble.cs, plus agent/appsettings.json - ship to a sister site unscanned. Re-running the assembled site patterns by hand over agent/ (excluding bin/obj) finds one hit, prose in agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj:108, which ADR-015 permits. The tree is clean; the gate would not notice if it stopped being clean. Note this is a gap in ADR-015's own stated scope, not a breach of it: the 2026-08-14 amendment names those six directories.

9. Missing Error Handling on External Calls

Verdict: REFUTED.

The plugins the finding names do not exist. ls plugins/ gives: applications, backups, computers, employees, endpoints, geenforce, guides, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, tools, usb, warranty. There is no ServiceNow plugin, no LDAP sync plugin and no email plugin; mail is core (shopdb/utils/mailer.py).

Every outbound HTTP call in the tree carries a timeout and an exception handler:

  • plugins/warranty/services/providers.py:129 and :157 - timeout=HTTP_TIMEOUT, except requests.RequestException raising WarrantyLookupError.
  • plugins/printers/services/zabbix_service.py:95 and :126 - timeout=self.REACHABLE_TIMEOUT / self.API_TIMEOUT, caught.
  • shopdb/utils/mailer.py:238 - timeout=10, caught, returns (False, error) and never raises.

10-12. Additional High Findings

The original text deferred these to a memory file rather than citing code: "inconsistent permission checking across plugins", "plugin state leakage via module-level globals", "missing database connection pool tuning".

The third is REFUTED. The pool is sized on purpose and env-driven: shopdb/config.py:20-22 (DB_POOL_SIZE 10, DB_MAX_OVERFLOW, DB_POOL_TIMEOUT 5) and :48-72, with the reasoning for the non-default pool_timeout written in place, and docker-compose.yml:172-193 documents the workers x (pool_size + max_overflow) arithmetic against MySQL's max_connections.

The other two were not re-checked, because neither cites a file. They should be restated with a file:line or dropped.

Medium Priority Findings (P2)

13. No Rate Limiting on API Endpoints

Verdict: HOLDS (narrowed). Login is limited; two kiosk surfaces are not.

Login rate limiting exists and is not "planned Q1 2027": shopdb/core/api/auth.py:35 _login_ratelimited(), keyed per host at :52, applied at :80, with tests/test_core/test_auth_ratelimit.py over it. Enrollment has its own limiter (plugins/endpoints/api/ratelimit.py, 300 per 60 seconds per IP applied at plugins/endpoints/api/enrollment.py:306).

What has none: the USB kiosk routes. grep for limiter or ratelimit across plugins/usb/ returns nothing, while plugins/usb/api/selfhosted.py:346-371 resolves a posted badge to an employee name and returns it. That is an unauthenticated, unthrottled badge-to-name lookup for any caller inside the configured kiosk CIDRs.

Both limiter counters live in per-process SimpleCache (plugins/endpoints/api/ratelimit.py:10-11), so at the shipped four gunicorn workers the effective budget is four times the configured maximum.

14. Frontend State Management Inconsistencies

Verdict: REFUTED.

grep -rn "vuex" frontend/src/ returns nothing, and frontend/package.json:28 declares "pinia": "^2.1.0". There is no Vuex-versus-Pinia split to standardise.

15. No API Versioning

Verdict: HOLDS.

grep -rn "api/v1" shopdb/ --include=*.py returns nothing; every route is mounted under /api/. This is a v1.0 decision, not a defect, and ADR-002 already carries the versioning story for the plugin contract. The 60-hour estimate has no stated basis.

16-22. Additional Medium Findings

Re-checked on 2026-09-18:

  • No content security policy headers: REFUTED as stated. shopdb/__init__.py:340 sets Content-Security-Policy: frame-ancestors 'self' on every response, plus X-Content-Type-Options, X-Frame-Options and Referrer-Policy at :334-337; uploads carry a stricter default-src 'none'; sandbox (shopdb/utils/uploads.py:13), pinned by tests/test_core/test_security_headers.py. The accurate, narrower statement is that there is no script-src policy, and the code says so in place and says why.
  • Missing health check endpoints: REFUTED. shopdb/core/api/dashboard.py:318 is GET /api/dashboard/health.
  • Database connection pool too small: REFUTED. See 10-12 above.
  • No request ID tracing: HOLDS. No request-id plumbing was found in shopdb/__init__.py.
  • No structured logging format: not re-checked.
  • Plugin discovery relies on a filesystem scan: HOLDS, by design. ADR-003 decides filesystem discovery under plugins/. "Slow" was never measured.
  • Frontend bundle size 2.1MB: UNVERIFIED. No build was run for this page.

Remediation Priority (revised 2026-09-18)

Finding Verdict What is actually left to do
Circular plugin dependencies REFUTED / narrowed Decide whether deferred cross-plugin imports must be guarded; guard agentauth.py:118 and policy_routes.py:44
Undeclared core imports REFUTED Nothing
Schema drift REFUTED / narrowed Align notification.py:158 with migration 0006; teach the naming gate to read sa.Column('...')
Plugin chain touches core tables HOLDS (new) Add a test that a plugin chain writes only owned tables
Migration chain validation STALE Nothing; tests/test_plugin_migrations.py is the gate
Implicit DB session sharing REFUTED Document the transaction pattern in PLUGIN-DEVELOPMENT.md
Frontend plugin imports STALE Nothing; gate passes
Site literals in plugins REFUTED / narrowed Extend the ADR-015 scan to agent/ and .cs
Missing external-call error handling REFUTED Nothing
Rate limiting HOLDS (narrowed) Throttle the USB kiosk badge lookup; decide whether per-process counters are acceptable at 4 workers
Vuex/Pinia split REFUTED Nothing
API versioning HOLDS v1.0 decision
Single points of failure UNVERIFIED Belongs in a deployment document

Testing Performed (2026-09-18 re-check)

# Every claim above was checked with a grep or a read on gh-wj at 35d4fcd41.
grep -rn "^from plugins\." plugins/ --include=*.py            # 0 hits
grep -rn "from shopdb\.\(core\|utils\|extensions\|services\)" \
  plugins/ --include=*.py                                     # 0 hits
grep -rPn "sa\.Column\(\s*'[a-z]+_[a-z_]+'" \
  plugins/*/migrations/versions/*.py migrations/versions/*.py # 0 hits
grep -rn "MACHINE_TYPES\|Bldg 300\|Bldg 400" . --include=*.py # 0 hits
grep -rn "vuex" frontend/src/                                 # 0 hits
grep -rn "api/v1" shopdb/ --include=*.py                      # 0 hits
bash scripts/check-naming-and-style.sh                        # exit 0

No test suite was run for this re-check; every verdict above is static.

References

  • ADR-001: Asset as Platform Contract (plugin isolation model)
  • ADR-003: Plugin Distribution (filesystem discovery)
  • ADR-008: Plugin Migration Ownership (linear chains, PLUGIN_TABLE_OWNERS)
  • ADR-013: Plugin Catalog and Lean Builds (frontend isolation)
  • ADR-015: Site-Specific Configuration (no literals in code)
  • ADR-Conformance-2026-09-18.md: the ADR-by-ADR result of the 2026-09-18 review, including the fact that the ADR set is not in this repository