Enabling a plugin could not make its routes live. Flask refuses
register_blueprint once the app has served a request, and each gunicorn worker
holds its own app object, so registering in the worker that served the toggle
would leave the others answering 404 for the same URL.
Worse, the window before the restart was a window in which workers disagreed
about reality. Four defects, all of which answered 200 while going wrong:
- Enabling a not-yet-installed plugin ALWAYS 500'd on a running app:
install_plugin ran the migrations, updated the registry, and only then called
register_blueprint and hit Flask's assertion. The toggle half-applied and
reported failure.
- Enabling one plugin could hang the entire site. @jwt_required does SELECT
users, so the request's transaction sits open holding a shared metadata lock;
the plugin's migration runs on its own Alembic connection and needs an
EXCLUSIVE lock on users for a foreign key. Each waits on the other, and MySQL
grants metadata locks in request order, so every later login queued behind the
pending exclusive. Observed twice as a total outage from one plugin toggle.
- Each worker rewrote the WHOLE registry file from a snapshot taken at boot, so
concurrent enables erased each other: seven enables persisted three, with
HTTP 200 for all seven.
- Dependency checks read that same stale snapshot, refusing a plugin whose
dependency another worker had just enabled.
Routes still require the process to restart; what changes is who initiates it.
Each worker fingerprints instance/plugins.json at boot and exits gracefully when
it changes, and the supervisor replaces it. That is the primitive gunicorn's own
--reload uses. Recycles are staggered per worker so the pool rolls rather than
dropping at once, and the exit is scoped to gunicorn: on Windows sys.exit() in a
daemon thread ends the thread rather than the process, so IIS keeps the manual
restart until that path is proven on the Windows VM.
The registry now merges under an exclusive lock and replaces atomically, saves
only the entry being changed, and re-reads before any decision is made from it.
ADR-020 records the decision; ADR-013 is amended where it said the UI must tell
the operator to restart. The restart requirement and the honesty requirement
both stand - only the manual step is removed, and only where it is proven.
(cherry picked from commit 9d6342b3ab)
Finding F12 of the architectural review, checked against the code and found
worse than reported: the dynamic component import cannot be rewritten by the
bundler, so the built file ships the unresolved path and a plugin tab answers
404 in every production build, staged or not. It works only under the dev
server. Nothing shipping is affected, because the one plugin that declares a
tab also registers the same page as an ordinary route, and that is the one
sites reach.
- The ADR draft now carries a "Constraints as implemented" section: only the
model catalog hub registers plugin tabs, registration happens when that hub
mounts so a tab is not deep-linkable and does not survive a refresh, and the
build finding above with the bundler's own warning. The two ways out are
stated as the open decision rather than taken.
- The hook's docstring reads as instructions to a plugin author, and its
example no longer names a path that does not exist at build time.
- A route declared for a hub that cannot render it is dropped with an error
naming the plugin, the hub and the hubs that do work, and the same is logged
at startup. Enabling the plugin still succeeds: a tab nobody can reach is no
reason to take a working backend offline.
- Tests hold the two halves together: no bundled plugin may declare a hub the
frontend does not implement, and the list in the server must equal the hubs
that really register routes, so the two cannot drift apart again.
Findings F2, F4, F5, F6, F7, F8, F9, F10, F11 and F13 from
wj-architectural-review.md. F1/F1a (the non-sargable LOWER pattern) and F3
(the warranty sync holding a pooled connection) are being handled separately
and are untouched here.
- A collector check-in that raced another for the same PC lost everything:
the unique index on the application fingerprint raised, nothing caught it,
and the whole check-in rolled back, discarding state, events and hardware,
answering the agent with form copy meant for a person. The insert now runs
in a savepoint, reconciles what the other transaction already wrote, one row
per savepoint so a second collision skips a row rather than the batch, and
the check-in commits with a warning. This is the shape the delete side of
the same function already had.
- Enrollment is deliberately unauthenticated, so it now carries a per-address
rate limit and a cap on pending requests, and the cap is a setting an admin
can see and change. A flood can no longer bury the approval queue or grow
the table without bound.
- The fleet alert poll no longer formats debug strings on every request.
- BusinessUnit is on the contract surface (0.34.0), so the endpoints plugin
stops reaching into a core table with raw SQL.
- A test walks every asset type descriptor and proves the endpoint it names
really refuses a caller without the declared permission.
- The plugin settings routes now need a permission; they were telling anyone
which modules a site runs and where its components live.
- The connection pool is sized explicitly with a short timeout, and the
arithmetic against the server's connection ceiling is written next to the
worker count instead of being left to defaults nobody chose.
- Fleet alerts carry both the correct lowercase keys and the old camelCase
pair. Trays older than 1.7.0 match keys case-sensitively and would have gone
quiet on every PC not yet upgraded; the old pair goes when they are.
- The dev server proxy points at localhost again, with environment variables
for the remote-box workflow, and one developer's address is out of the
server certificate page.
Per-PC credentials (replaces the 09-14 client-certificate attempt):
- POST /api/endpoints/enroll issues a credential to a PC that matches exactly
one computer by hostname and BIOS serial and has none yet. Anything else
(unknown or duplicate hostname, serial mismatch, already enrolled) waits as
a request an admin approves or rejects. The agent's persisted nonce binds an
approved request to the PC that asked.
- One auth helper serves every agent route. With X-Endpoint-Credential the
asset comes from the credential and a request naming another PC gets 403
ASSET_MISMATCH. The shared key still works for PCs that have not enrolled,
is refused (401 SHARED_KEY_REFUSED) for any PC that has, and can be switched
off. A revoked credential gets 401 CREDENTIAL_INVALID so the agent
re-enrolls.
- The core collector accepts the credential for the computers, printers and
endpoints collectors through two new plugin hooks exposed on shopdb.api
(contract 0.33.0), and refuses the shared key for enrolled PCs.
- Admin API: enrollment list, approve, reject, revoke (un-enroll), settings.
Migration API: per-PC readiness with blockers, and upgrade waves that
package the agent once and queue the existing agent upgrade per PC.
- Migration 0023 creates endpointcredentials, endpointenrollrequests and
endpointmigrationwaves, drops the certificate columns from assets where
test databases have them, and renames app_fingerprint to appfingerprint.
0021 and 0022 keep their ids but no longer alter the core assets table.
- Removed: client_cert.py, certificate renewal, the unauthenticated
/test/client-cert route, nginx client certificate forwarding and the agent
certificate script. The client-cert header could be forged on IIS, the key
was readable by users, and the shared key remained a fallback.
- GET /settings/server-cert returns the SPKI SHA-256 pin for GE-Enforce.
Review fixes:
- report_command_result used g without importing it; every result was a 500.
- collect_log refuses setup answer files, appsettings, agent directories and
key files at enqueue.
- Fleet alerts: alerttimeout and alertfleet agree with the migration, the
scope is validated and a malformed scope no longer reaches the whole fleet,
and the ack race no longer raises NameError.
- A refused break-glass override leaves no hold and no audit row.
- Agent upgrade checks locks and permissions before repackaging, and writes
the package atomically.
- Server certificate upload validates the pair before replacing files;
generation no longer builds an openssl config from user input; audit rows
are actually written.
- Health evaluation runs in a savepoint so a caught error cannot lose the
check-in.
- Required-app policies are found by appid, survive renames, and deleting an
app deactivates policies that still name it. The UI and API now say such a
policy only takes effect inside a business policy.
- Backfill handles duplicate fingerprints and rolls back a failed batch.
- Plugin settings routes skip a plugin whose hook raises.
- The setup token is no longer printed to the log.
- The naming gate checks top-level files again and handles odd file names.
- Removed root scratch scripts, one of which carried a token.
- Rewrote the fleet alert scheduling and app delta tests on real fixtures.
Bugfix: Fix AttributeError in /api/plugins/settings-routes endpoint
- Changed get_enabled_plugin_instances() to get_all_plugins()
- Added explicit enabled check with pm.registry.is_enabled()
- Test failure: test_no_route_answers_a_nonsense_filter_with_a_server_error
Optimization: Enable parallel test execution (Phase 1 of pytest speedup)
- Added pytest-xdist configuration to pytest.ini
- Enabled -n auto (auto-detect CPU cores, typically 8 workers)
- Added slow marker for future test categorization
- Expected speedup: 4-8x (20 min -> 2.5-5 min)
Root cause: 2857 tests recreate entire schema per test (0.4-0.6s overhead each)
- Phase 2 (savepoint isolation) will achieve 80x total speedup
- See docs/PYTEST-OPTIMIZATION-PLAN.md for full analysis
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Plugins can now register their settings pages dynamically via get_settings_routes()
hook, eliminating hardcoded imports that break lean builds.
Backend:
- Add get_settings_routes() hook to BasePlugin (shopdb/plugins/base.py)
- Implement in ComputersPlugin for Operating Systems management
- Add /api/plugins/settings-routes endpoint to expose plugin routes
Frontend:
- ModelCatalogHub fetches plugin routes and dynamically adds them via router.addRoute()
- Tabs render dynamically from pluginTabs ref
- Components lazy-load when clicked
Benefits:
- Lean builds work without manual route exclusions
- Plugins self-contain their UI extensions
- No core code changes needed when adding plugin settings pages
- Follows plugin architecture best practices
Fixes TODO from commit a03fd3de4f
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Flask + Vue rewrite of the classic-ASP shopdb, built as a plugin
framework so sister GE Aerospace sites can adopt it. Product version
0.7.0-pre-0.8.0, plugin contract 0.11.0, 966 backend tests.
Key entry points:
- docs/INSTALL-WINDOWS-IIS.md - tested Windows + IIS install runbook
(own-site and subpath deployment methods)
- docs/PILOT-DEPLOY.md - production pilot: stand up + import + verify
- docs/IMPORT-API.md - schema-agnostic legacy data import contract
- scripts/site_imports/wjf/ - West Jefferson reference import loader
- docs/adr/ - twelve accepted architecture decision records
