fa6b99bf919c8a880015b4a884b9428cb8067fe3
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
fa6b99bf91 |
Finish a restore without a browser, from the set that was asked for
Closes the three defects in the imported restore path, at Ben's request. Each was proven against a real stack before and after: two backup sets with different data and different instance files, then a restore of the OLDER one, which is the case that was broken. A restore no longer depends on a tab staying open. The backup sidecar has no Flask, so it loads the dump and leaves a sentinel for the api container to run the migrations. The only thing that ever acted on that sentinel was the poll loop of the modal that queued the restore, so closing the tab - or letting a token expire - during the minutes a load takes left the data at the old revision under new code, every query on a newer column failing, and nothing retrying. A watcher in the api container now claims and runs it. The endpoint no longer runs the upgrade at all; it only makes sure something is pending, which is also how a failed upgrade is retried. That takes Alembic out of the request path, where it could outrun the gunicorn worker timeout and be killed mid-chain, leaving a half-applied revision the next attempt trips over. The claim is a rename, because rename(2) is atomic: several workers and a browser poll can race the same sentinel and exactly one wins. A failed upgrade parks itself as .upgrade-failed rather than re-arming, since putting it back would rerun the same broken chain on every poll forever. Instance files come from the set being restored. The restic call asked for 'latest' regardless of which set was named, so restoring anything but the newest paired that set's DATABASE with the NEWEST snapshot's uploads and plugins.json - a mismatch nothing downstream could detect. Now scoped by the set's own tag, which backup.sh has always written. The tree is replaced rather than restored over, so files created after the snapshot are gone afterwards: a restore has to put the site back to the snapshot, not merge into it. Done as a staged swap rather than 'restic restore --delete --target /', because that aims a delete pass at the container root and the cost of getting its filter wrong is the whole filesystem. Nothing destructive happens before a copy is taken. The dump carries DROP TABLE per table with no transaction around it, and the instance tree is replaced wholesale, so a wrong set or a failure part-way through left nothing to go back to. restore.sh now writes a pre-restore-<stamp>/ set - dump, instance tarball and a README saying what it is and how to load it - and refuses to touch anything if that copy cannot be written. Two more found while proving the above, both of which stopped the overlay working at all: * Every script under scripts/backup/ was committed non-executable, while every other .sh in the repo is 755. The compose entrypoints are /opt/backup/watch.sh and backup.sh from a read-only bind mount, so the backup and watcher containers could not start from a clean checkout on any machine, and nothing could fix it at runtime. * Dockerfile.backup could not build without the corporate CA staged. A plain COPY of the gitignored cert fails outright; the trailing-glob form does too, because BuildKit fails a COPY whose pattern matches nothing - which is exactly the case being handled. Copying the directory and testing for the file inside it is the form that works both ways. Verified by building the image with no cert present. Suite 2600 passed / 9 skipped. The finalizer has unit tests covering the atomic claim, the parked failure and the endpoint no longer running the upgrade. Scope: product |
||
|
cproudlock
|
83190422c9 |
Act on the review of the wilm import
Six findings from a four-agent review of the imported work, plus the contract bump the import owed. Nothing here changes what the imported features do. The contract version was not bumped for a change that widens what a plugin may declare. The asset-panel vocabulary gained two tab groups (serviceprocess, reports), a datetime column format, a map.body key and a tabs section that renders as a table. 0.29.0 bumped for exactly this class of change when the optional group was added, so this is 0.31.0 with a ledger row. All four are optional and resolve server-side when absent, so a plugin declaring none of them is unaffected. Site Backups now answers honestly where it has nothing behind it. The overlay is a compose profile: a separate container writes sets onto a shared volume and the api container only reads them. A Windows/IIS install runs no such container and the default root resolves to a drive-letter path nothing creates, so Back Up Now either failed on PermissionError or reported a run queued that would never happen, under a banner reading "No verified backup yet" - which describes a backup that failed, not a feature that was never installed. The three write routes refuse with the same 404 shape the configuration-policy routes use for a disabled feature, the list and status routes report whether a store exists, and the page says so instead of offering buttons. Two listAll callers still read a response that is not there. listAll resolves to a plain array through fetchAllPages, so response.data.data throws: PC Types settings rendered empty, and in Configuration Policies the throw happened inside the shared try, so PC types, applications, the action catalog and monitors were all left unassigned by one bad read. The agent package upload cannot succeed on IIS as shipped. MAX_CONTENT_LENGTH went to 256 MB to admit the ~69 MB tray binary, but IIS request filtering caps a body at 30,000,000 bytes and answers 413 before waitress sees the request, so web.config needs the matching requestLimits. The backup image could not be built by anyone outside a TLS-intercepting network. Dockerfile.backup did a plain COPY of a gitignored corporate root CA, and a COPY of a missing file fails the build. The trailing glob makes it a no-op when the cert is absent. endpointstates.domain was the one new string column written without a clamp. Under STRICT_TRANS_TABLES an over-long value raises Data too long and aborts the whole check-in, so a single malformed report would stop a machine reporting telemetry and events entirely. Not fixed here, and left with upstream: the restore path finalizes its schema upgrade only from a browser poll loop, so closing the tab mid-restore leaves the data behind the code; it restores instance files from the newest snapshot rather than the set that was asked for; and it takes no safety dump before the destructive load. All three are in the Docker-only overlay, which this site does not run, and none can be validated here. Scope: product |
||
|
cproudlock
|
6e36df841e |
Import the wilm endpoint telemetry, PC Detail rework and backup overlay
Ports 37 of Benjamin Baker's commits from the wilm branch, mirrored from the
corporate GitHub at
|