7f5210f2a3
Some checks failed
The full suite caught it: docs/INDEX.md routes by this metadata, so a page
without it is invisible to the router that exists to make the corpus findable.
Failed on gh-main and would have failed CI.
(cherry picked from commit 0ded9207ab)
101 lines
4.8 KiB
Markdown
101 lines
4.8 KiB
Markdown
<!-- docmeta
|
|
purpose: explanation
|
|
audience: developer
|
|
status: current
|
|
summary: Why enabling a plugin no longer needs a manual restart, and where it still does.
|
|
loadwith: adr/ADR-013-plugin-catalog-and-lean-builds.md
|
|
-->
|
|
|
|
# ADR-020: Worker self-recycle when the enabled-plugin set changes
|
|
|
|
Status: Accepted
|
|
Date: 2026-09-20
|
|
Amends: ADR-013 (see "Relationship to ADR-013")
|
|
|
|
## Context
|
|
|
|
Enabling a plugin could not make its routes live in a running process. Flask
|
|
refuses `register_blueprint` once the app has served its first request, and even
|
|
without that rule each gunicorn worker holds its own app object, so registering
|
|
in the worker that served the toggle would leave the other workers answering 404
|
|
for the same URL. The operator therefore had to restart the app by hand.
|
|
|
|
That restart was not merely inconvenient. Because every worker also holds its own
|
|
`PluginRegistry`, loaded once at boot, the window between a toggle and a restart
|
|
was a window in which workers disagreed about reality:
|
|
|
|
- a dependency check refused a plugin whose dependency another worker had just
|
|
enabled
|
|
- a whole-file registry write from a stale snapshot erased other workers'
|
|
changes, and the API answered 200 while doing it
|
|
- a health check that compared the registry against loaded plugins reported "no
|
|
restart needed" while the routes 404'd
|
|
|
|
## Decision
|
|
|
|
Flask's constraint stands. A plugin toggle still requires each serving process to
|
|
restart. What changes is who initiates it.
|
|
|
|
1. Each worker fingerprints `instance/plugins.json` (content hash) at boot.
|
|
2. After serving a response, if the fingerprint has changed, the worker exits
|
|
gracefully and its supervisor starts a replacement.
|
|
3. `instance/plugins.json` is the single cross-worker source of truth for the
|
|
enabled set. It is written only through the locked merge-write in
|
|
`PluginRegistry._save`. Per-worker registries are caches and must be
|
|
refreshed from disk before any decision is made from them.
|
|
4. Recycles are staggered per worker, so the pool rolls rather than dropping at
|
|
once, and are bounded by the supervisor's graceful timeout.
|
|
5. This applies only where a supervisor will return the process AND the exit
|
|
path has been exercised. Today that is gunicorn. It never applies under the
|
|
development server.
|
|
6. `/setup/health` describes the process that answered it. Callers must not
|
|
infer fleet state from a single reply; confirm across several distinct
|
|
workers.
|
|
|
|
## This is a convergence mechanism, not a security control
|
|
|
|
Disabling a plugin has never stopped its code being imported. `GET /api/plugins`
|
|
instantiates every plugin class on disk, core request handlers import plugin
|
|
models gated on `ImportError` rather than on enabled state, and ADR-013:23 says
|
|
plainly that "Disabled is not absent". Containment comes from the lean build
|
|
(ADR-013) and from signature enforcement (PLUGIN-SIGNING.md).
|
|
|
|
A stale worker briefly serving a disabled plugin's routes is therefore a user
|
|
experience artifact, not a breach, and nothing here should be cited as though it
|
|
were a boundary.
|
|
|
|
## Relationship to ADR-013
|
|
|
|
ADR-013 states that an enable "takes full effect only on restart, and the UI says
|
|
so. There is no 'install button' that pretends otherwise", and that "the
|
|
marketplace UX is honest about it rather than working around it".
|
|
|
|
That principle is kept, not reversed. The restart still happens; the process
|
|
still reloads through the ordinary boot path, so verification and the import
|
|
guard apply exactly as on a manual restart. What is removed is the manual step,
|
|
on the one runtime where the mechanism is proven.
|
|
|
|
The honesty requirement is now explicit rather than implicit:
|
|
|
|
- Where a recycle will happen, the UI holds the plugin at "starting" until
|
|
several distinct workers confirm it is live, and only then says "ready".
|
|
- Where a recycle will NOT happen, the UI keeps the old message and the operator
|
|
restarts. Saying "restarting" on a runtime that does not is worse than the
|
|
manual step it replaced.
|
|
|
|
## Consequences
|
|
|
|
- A toggle takes effect in seconds without operator action, on gunicorn.
|
|
- Worker disagreement about the enabled set becomes transient rather than
|
|
standing, which removes a class of lost-update and stale-read bugs.
|
|
- Windows/IIS (waitress behind HttpPlatformHandler) is single-process and
|
|
unproven here, so it keeps the manual restart. Adding it requires testing on
|
|
the Windows VM and handling HttpPlatformHandler's `rapidFailsPerMinute`, which
|
|
stops relaunching after repeated exits.
|
|
- A worker recycling is indistinguishable from a worker dying, so the supervisor
|
|
must be configured to replace workers. All shipped compose files and the
|
|
Kubernetes manifest run gunicorn and satisfy this.
|
|
- Migrations run on their own connection with a bounded `lock_wait_timeout`, so
|
|
a DDL that cannot take its metadata lock fails that one enable instead of
|
|
queueing every later reader behind it.
|