From 1d73bd477ebc145c14e58f2dc428e74a7350a490 Mon Sep 17 00:00:00 2001 From: cproudlock Date: Tue, 4 Aug 2026 20:54:05 -0400 Subject: [PATCH] Document what future Windows releases look like, for operators and for builders Two audiences, two documents. Both were only in people's heads. UPDATES-WINDOWS.md is for whoever runs a server: updates arrive as one self-contained exe, an update takes two to four minutes, the site is down for that time, .env and data and any hand-edited web.config are kept, unticking a feature never removes it, the database is backed up and verified first, and a downgrade is refused because migrations only go forwards. It covers both kinds of security release, application and third-party, and explains that the CycloneDX inventory staged on every server is what answers a published vulnerability question. It also says plainly that the exe is not signed and the checksum is the integrity check to rely on today. It answers one question the existing docs did not address at all: the effect on other sites sharing the same IIS server. The application pool is isolated and the configuration is scoped to its own path, so other sites keep their own handlers. What IS shared gets named rather than glossed: installing the IIS modules and writing server-level configuration recycles application pools across the server, which can drop requests in flight and clears in-memory session state, though IIS is never stopped and no iisreset is issued. The two IIS modules and the single permitted rewrite server variable are machine-wide and stay behind on uninstall, deliberately, since another site may have come to depend on them. The bundled database option collides on port 3306 with an existing MySQL. RELEASING-WINDOWS.md is for whoever builds releases: the three kinds of change and the commands for each, why bundle-lock.json must be committed, the two dependency traps that have each already cost a release, which generated files must never be hand-edited, and the pre-release checks. It records the two known gaps honestly - no code signing, and compiling still requires Windows and a person. UPGRADE.md and OPERATE-WINDOWS.md link to the operator document. --- docs/OPERATE-WINDOWS.md | 8 ++ docs/RELEASING-WINDOWS.md | 167 ++++++++++++++++++++++++++++++++ docs/UPDATES-WINDOWS.md | 199 ++++++++++++++++++++++++++++++++++++++ docs/UPGRADE.md | 4 + 4 files changed, 378 insertions(+) create mode 100644 docs/RELEASING-WINDOWS.md create mode 100644 docs/UPDATES-WINDOWS.md diff --git a/docs/OPERATE-WINDOWS.md b/docs/OPERATE-WINDOWS.md index dc8e61b..29b209a 100644 --- a/docs/OPERATE-WINDOWS.md +++ b/docs/OPERATE-WINDOWS.md @@ -231,3 +231,11 @@ than a configuration mismatch. If the bundled MySQL was installed, its generated root password was written once to `C:\ProgramData\ShopDB-Flask\mysql-root-password.txt`. **Move it into your password manager and delete that file.** It cannot be recovered. + +## See also + +- [UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) - what future updates, bug fixes and + security releases will look like, including downtime and the effect on other + sites on the same IIS server +- [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore +- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - installing a new site diff --git a/docs/RELEASING-WINDOWS.md b/docs/RELEASING-WINDOWS.md new file mode 100644 index 0000000..c8c26e1 --- /dev/null +++ b/docs/RELEASING-WINDOWS.md @@ -0,0 +1,167 @@ +# Building and releasing the Windows installer + +For whoever builds releases. If you run a server rather than build releases, see +[UPDATES-WINDOWS.md](UPDATES-WINDOWS.md). + +Every release is one self-contained `.exe`. It carries the Python runtime, a +hash-checked wheelhouse, MySQL, the IIS modules and the application, so nothing +is fetched from the network at install time. That is the point: sites are +air-gapped. + +## What you need + +- A checkout of this repository. +- Inno Setup 6.6.0 or newer, on Windows. Compiling needs Windows; staging the + bundle does not. +- PowerShell, for the bundle lock tools. +- `uv`, only when dependencies change. + +## The three kinds of change + +Almost every release is the first kind. + +### 1. Application change: bug fix, feature, no new dependency + +```bash +# bump __version__ in shopdb/__init__.py first +./deploy/windows/installer/build-installer.sh +``` + +Then on Windows, in `deploy/windows/installer`: + +``` +iscc ShopDBFlask.iss +``` + +`build-installer.sh` restages the application, rebuilds the web interface for +the chosen feature set, regenerates `plugins.iss` and `version.iss`, and +re-verifies the third-party payload against `bundle-lock.json`. The lock is +untouched, because nothing third-party changed. + +### 2. A dependency is added, removed or moved + +The only case that touches the lock. + +```bash +# edit requirements.in, then +uv pip compile requirements.in --universal --generate-hashes -o requirements.txt + +# fetch the wheel for the target runtime, into the wheelhouse +pip download == -d deploy/windows/installer/bundle/wheels \ + --only-binary=:all: --platform win_amd64 --python-version 314 --no-deps +``` + +Then regenerate and commit the lock: + +```powershell +pwsh ./refresh-bundle-lock.ps1 # review the change +pwsh ./refresh-bundle-lock.ps1 -Yes # write it +``` + +**Commit `bundle-lock.json`. That commit is the review.** The lock records every +third-party file and its SHA-256; the installer refuses to run if the payload it +carries does not match, so an unreviewed substitution cannot reach a server. + +Two traps, both of which have already cost a release: + +- `--universal` is not optional. A resolve done only for the host platform drops + packages that exist only on Windows, and the install then fails hash checking + on a package with no entry. +- Declare **every** runtime import in `requirements.in`, including ones that + already work locally. `packaging` reached this project only as a test + dependency, so the full suite passed while a production virtual environment, + which has no test dependencies, could not import the application at all. + `tests/test_runtime_dependencies.py` is the gate for this. + +### 3. A new plugin + +Ordinary plugin work, plus three installer-specific steps: + +1. **Stage it.** Add the plugin to the profile you build from. `plugins.iss` is + generated from what is actually in the bundle, so the wizard offers it with + no edit to the installer script. +2. **Decide whether it is ticked by default.** `PluginDefault()` in + `ShopDBFlask.iss` is an exclusion list: a new plugin defaults to **ticked** + unless you name it there. This is the one manual edit. +3. **Register its migrations.** Update `PLUGIN_TABLE_OWNERS` and + `EXPECTED_HEAD_REVISION`, and give the plugin its own migration chain + (ADR-008). Existing sites pick up its tables through `plugin upgrade-all`; + sites that do not tick it have them pruned (ADR-014). + +## Version numbers + +`build-installer.sh` reads `__version__` from `shopdb/__init__.py` and writes +`version.iss` from it. Do not edit `version.iss`: a hardcoded version in the +installer script had already drifted two minor versions from the code. + +Bump the version for every release you hand out. The installer compares versions +and: + +- refuses a build **older** than what is installed, because migrations only go + forwards; +- warns and continues on an **equal** version, treating it as a repair. + +Equal-version rebuilds are useful while testing and are a poor idea in the +field, since the server cannot then tell you what it is running. + +Follow ADR-007 for what each part means. + +## Do not hand-edit generated files + +Regenerate these; changes are overwritten without warning: + +- `version.iss` - from `shopdb/__init__.py` +- `plugins.iss` - from the plugins actually present in the bundle +- `requirements.txt` - from `requirements.in` via `uv pip compile` +- `bundle-lock.json` - via `refresh-bundle-lock.ps1` + +`waitress` and `tzdata` were once hand-added to `requirements.txt` and vanished +on the next compile, taking the Windows runtime with them. + +## Before you hand a build out + +```bash +python -m pytest -q # full suite +./scripts/check-naming-and-style.sh # naming and style +``` + +The suite includes gates worth knowing about: + +- `tests/test_runtime_dependencies.py` - every runtime import is declared +- `tests/test_bundle_lock.py` - the lock covers the payload +- `tests/test_docs_publishable.py` - `docs/` carries no internal references, + because it is published + +`build-installer.sh` refuses to finish if the payload does not match the lock. +That check is not advisory; do not work around it. + +## Publishing + +Alongside the `.exe`, publish: + +- a `.sha256` file, so the operator can verify what they received; +- release notes covering what changed and anything needing attention; +- the version in the filename, so a server's build is identifiable on sight. + +The compiled installer stages a CycloneDX inventory (`sbom.cdx.json`) onto every +server, which is what answers a security question about a published +vulnerability without anyone guessing. + +## Known gaps + +**The installer is not code-signed.** Every install shows an unknown-publisher +warning, and the SHA-256 is the only integrity check. This is the significant +remaining gap before wider distribution: a checksum published next to the file +protects against corruption, not against someone who can write to that location. + +**Compiling requires Windows.** `build-installer.ps1` exists so the whole +process can run on a Windows workstation. Nothing about it runs in CI, so a +release is a deliberate act by a person. + +## See also + +- [UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) - what operators should expect +- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - installing a new site +- [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md) - running a site +- `docs/adr/` - ADR-007 versioning, ADR-008 plugin migrations, ADR-013 and + ADR-014 lean per-site builds diff --git a/docs/UPDATES-WINDOWS.md b/docs/UPDATES-WINDOWS.md new file mode 100644 index 0000000..366b0c4 --- /dev/null +++ b/docs/UPDATES-WINDOWS.md @@ -0,0 +1,199 @@ +# What to expect from updates (Windows sites) + +For the people who run a ShopDB-Flask server. It covers how updates arrive, what +they do to your data and your server, how long they take, and what happens to +anything else running on the same machine. + +If you are the person building releases, see +[RELEASING-WINDOWS.md](RELEASING-WINDOWS.md). + +## Updates arrive as one file + +Every release is a single `.exe`, the same kind of file you used to install. +There is no patch, no separate updater, and no download step during the install: +everything the server needs is inside that one file, including the Python +runtime, all library code and the application itself. + +To update, run the newer `.exe` on the server as Administrator. That is the +whole procedure. + +The installer works out for itself that this is an update rather than a first +install, and skips what is already correct. An update typically takes two to +four minutes, most of which is the database migration. + +## What an update changes, and what it leaves alone + +Changed: + +- The application code and the web interface. +- The database schema, brought forward by migrations. +- The Python runtime and libraries, but only when that release moves them. + +Left exactly as they are: + +- `.env`, which holds your database connection and secret keys. +- Your data. Updates migrate the schema; they do not reset or reload content. +- `web.config`, if you have edited it. The installer only ever repairs a + specific fault in it that older builds created, and copies the file aside + first when it does. +- Which features are switched on. The feature list opens showing what this site + already has. +- Uploaded files and anything under `instance\`. + +Unticking a feature during an update does **not** remove it. Adding is a tick; +removing is a deliberate, separate step. This is so an upgrade can never quietly +delete a feature and its data. + +## Downtime + +The site is down for the length of the update, so two to four minutes. The +installer stops the application pool before replacing files, because Windows +will not let it overwrite files a running process holds open, and starts it +again afterwards. + +There is no reboot. If a release ever needs one, the installer says so rather +than restarting the machine itself. + +## Your data is backed up first + +Before applying migrations, the installer takes a database backup and checks the +dump is readable. If a migration fails, it restores from that backup and tells +you. + +This depends on `mysqldump` being present. Confirm it once, before your first +update: + +```powershell +.\shopdb-admin.ps1 check +``` + +Without it the update still runs, but the pre-update backup is skipped, and that +is precisely the backup you would want if a migration went wrong. + +Afterwards: + +```powershell +.\shopdb-admin.ps1 status +.\shopdb-admin.ps1 verify +``` + +## Going backwards is refused + +Installing an older build over a newer one is blocked outright. Once migrations +have moved the schema forward, older code cannot read it, and the failures are +difficult to unpick. + +To go back you restore a backup taken before the update. Keep the previous +`.exe` until you are satisfied with a release. + +## Will an update affect other sites on the same IIS server? + +Mostly no, and the exceptions are listed here rather than glossed over. + +**Isolated from other sites:** + +- The application runs in its own application pool under its own identity, so a + crash or a memory leak cannot reach another site's pool. +- Its configuration lives in its own folder and applies only to its own URL + path. A parent site's own pages, including classic ASP, keep their existing + handlers. +- File permissions are granted on the application folder only. + +**Shared, and therefore worth knowing about:** + +- **A brief application pool recycle across the server.** Installing the IIS + modules and writing server-level configuration causes IIS to reload its + configuration, which recycles application pools. Requests in flight at that + moment can be dropped, and any session state other sites hold in memory is + lost. It is a few seconds and there is no service outage: IIS itself is never + stopped and no `iisreset` is issued. +- **Two IIS modules are installed machine-wide** the first time: + HttpPlatformHandler and URL Rewrite. Both are standard Microsoft modules. They + do nothing to a site that does not reference them, and if a site already uses + URL Rewrite its rules are untouched. +- **One rewrite server variable is permitted machine-wide**, + `HTTP_X_FORWARDED_FOR`, so the application can see real client addresses + instead of the loopback address. This grants exactly that one variable rather + than opening the section up. +- **A Microsoft C++ runtime** may be installed, which is shared and backwards + compatible. +- **The bundled database option installs MySQL on port 3306.** If this server + already runs MySQL, choose the existing-database option instead. Two servers + will collide on that port. The wizard asks before doing anything. + +Removing ShopDB-Flask takes away its own site, application, pool, folder and +firewall rule. It deliberately leaves the shared IIS modules in place, because +another site may have started depending on them. + +If your server hosts something critical, schedule updates in a maintenance +window for the pool recycle, not for the application itself. + +## Security updates + +Two kinds reach you, both as an ordinary `.exe`. + +**Application fixes** are built from the source and shipped like any other +release. + +**Third-party fixes** cover the Python runtime, the libraries, MySQL and the IIS +modules. These are pinned to exact versions and checked by cryptographic hash at +install time, so a release contains precisely the versions it claims and nothing +substituted. When one of them publishes a fix that affects this application, it +is picked up and a new release is issued. + +Each release ships a machine-readable inventory of every third-party component +and its version, installed on the server as `sbom.cdx.json`. If your security +team asks whether you are exposed to a published vulnerability, that file +answers it without anyone guessing. + +An update that is only a dependency bump is still worth taking: the version +number moves and the application behaviour does not. + +## Check the file before running it + +Each release publishes a SHA-256 checksum beside the `.exe`. Verify it: + +```powershell +certutil -hashfile ShopDBFlask_Installer_.exe SHA256 +``` + +Compare with the published `.sha256` file. + +Windows will warn about an unknown publisher, because the installer is not yet +code-signed. The checksum is the integrity check to rely on today. Get the file +from the agreed location rather than from mail or a message. + +## Version numbers + +Three parts, for example `0.7.0`: + +- The last part changes for bug fixes and security fixes. Nothing you use + behaves differently. +- The middle part changes for new features. Existing features keep working. +- The first part changes for something that needs you to read the notes first. + +Before 1.0 the middle number can still bring changes that need attention. Read +the release notes for those. + +## If an update fails + +The installer stops at the first problem rather than continuing, and says what +failed and what to do. Nothing is left half-applied: either the change is +complete or it is rolled back, and the log records everything either way. + +The log is at: + +``` +C:\ProgramData\ShopDB-Flask\logs\shopdb-install-.log +``` + +Re-running the same `.exe` is safe and picks up from where it stopped. If it +fails again, send that log with your report; it names the failing step, the +exit code and the relevant output. + +## See also + +- [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md) - day-to-day running +- [UPGRADE.md](UPGRADE.md) - upgrade notes across all deployment types +- [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore +- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - first-time install diff --git a/docs/UPGRADE.md b/docs/UPGRADE.md index 8fb8399..716d453 100644 --- a/docs/UPGRADE.md +++ b/docs/UPGRADE.md @@ -116,6 +116,10 @@ PY Run a newer installer `.exe` over the existing install. That is the whole procedure - none of the manual steps above apply. +[UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) is the operator-facing version of this: +downtime, what is and is not touched, security updates, and the effect on other +sites sharing the same IIS server. + It backs the database up first and verifies the dump, applies the core and plugin migrations, restores from that backup if a migration fails, and refuses to install an older build over a newer one. Your `.env`, your data and your