diff --git a/deploy/windows/installer/README.md b/deploy/windows/installer/README.md index 7debc62..a48a80c 100644 --- a/deploy/windows/installer/README.md +++ b/deploy/windows/installer/README.md @@ -19,27 +19,122 @@ so a separate repo would drift out of step with the thing it installs. | `build-installer.sh` | Stages the bundle from a site profile. | | `make-branding.py` | Generates wizard artwork and the icon from `frontend/public/*.svg`. | | `*.bmp`, `shopdb.ico` | Generated artwork, committed so a Windows build box needs no Python. | +| `build-installer.ps1` | The same build, natively on Windows. No Bash needed. | +| `bundle-lock.json` | The exact third-party payload this installer ships. Reviewed by commit. | +| `bundle-lock.ps1` | Creates and checks that lock. Also runs on the target server. | +| `refresh-bundle-lock.ps1` | Regenerates the lock, after showing what changed. | +| `verify_bundle_lock.py` | The same check for the Bash builder, so Linux needs no pwsh. | ## Building +Two builders, same result. Use whichever machine you are on - `build-installer.ps1` +does the whole job natively so a Windows work PC needs no Bash. + ```bash -# 1. Stage the bundle for a site's plugin set. +# Linux / WSL ./build-installer.sh ../../site-profile.example.json +``` -# 2. Add the pieces that cannot be built on Linux: -# bundle/wheels/ ~47 cp314 win_amd64 wheels, built ON Windows: -# pip download -r requirements.txt -d wheels --only-binary=:all: -# bundle/python/ python-3.14.x-amd64.exe -# bundle/httpplatformhandler/ httpPlatformHandler_amd64.msi -# bundle/mysql/ mysql-8.0.x-winx64.msi (bundled-database option only) +```powershell +# Windows +.\build-installer.ps1 -Profile ..\..\site-profile.example.json +``` -# 3. Compile on Windows. +Both stage the app tree, build the SPA twice, write `plugins.iss`, copy the +installer scripts **from this directory**, and then verify the third-party +payload against `bundle-lock.json`. They exit non-zero if it does not match. + +The payload itself is added by hand, and no longer needs Windows to produce: + +``` +bundle/wheels/ pip download -r requirements.txt --only-binary=:all: \ + --platform win_amd64 --python-version 314 \ + --implementation cp --abi cp314 -d wheels +bundle/python/ python-3.14.x-amd64.exe +bundle/httpplatformhandler/ httpPlatformHandler_amd64.msi +bundle/urlrewrite/ rewrite_amd64.msi (client-IP rule; see below) +bundle/mysql/ mysql-8.0.x-winx64.msi (bundled-database option only) +``` + +Then compile on Windows: + +``` iscc ShopDBFlask.iss ``` +**Inno Setup 6.6.0 or newer.** The wizard uses the built-in `windows11` custom +style, which earlier versions reject. The script fails at compile time with that +sentence rather than with a bare "WizardStyle is invalid". + The wheelhouse is **cp314-locked**. A different Python minor version means a different wheelhouse; the installer will not use a Python it did not install. +## What is pinned, and where + +Three layers, because no one of them covers the whole problem. + +| Layer | Covers | Enforced | +|---|---|---| +| `requirements.txt` sha256 per package | every wheel is genuinely what upstream published | `pip --require-hashes` at install; aborts on mismatch | +| `bundle-lock.json` | the EXACT payload: wheels, Python installer, MSIs | both builders, and again on the server before anything runs | +| git | the application tree | code review | + +`--require-hashes` alone is not enough. pip lists every artifact of a pinned +version - `cffi 2.1.0` has 100 hashes - so it proves the wheel is genuine, not +that it is the wheel this bundle was built and tested with. It also ignores +extra files in the wheelhouse, and says nothing about the Python installer or +the MSIs, all of which run as SYSTEM on the target server. + +So `bundle-lock.json` records an exact file set with a sha256 and a byte size +each, and verification is **set equality**: a missing file, an unexpected extra +file, or changed content all fail. There is no install-time override. + +The lockfile is `--universal`, so one file serves Linux (dev, Docker, CI) and the +Windows wheelhouse. A Linux-only resolve had silently omitted `colorama`, a +win32-only dependency of `click` - which in hash-checking mode is a hard error +rather than a quiet omission. + +### Changing what ships + +```powershell +.\refresh-bundle-lock.ps1 # show what changed, write nothing +.\refresh-bundle-lock.ps1 -Yes # write it +``` + +Then **commit `bundle-lock.json`**. That commit is the review - it is the only +place a change to what runs as SYSTEM on a customer's server becomes visible to +a human. `refresh-bundle-lock.ps1` refuses to overwrite an existing lock until +you have seen the diff, for that reason. + +To stage a bundle before its lock exists: `ALLOW_UNLOCKED=1` (Bash) or +`-AllowUnlocked` (PowerShell). Bundles built that way must not be shipped. + +Verifying a live server, months later and offline: + +```powershell +shopdb-admin.ps1 verify +``` + +## Client IP addresses + +IIS does not set `X-Forwarded-For` on its own, and HttpPlatformHandler connects +from loopback. Without a rule, **every client reads as 127.0.0.1** - so the +GE-Enforce IP allowlist, the dashboard's visitor-location lookup and per-host +login rate limiting all stop working, silently. + +The wizard asks, because the two answers are mutually exclusive: + +- **Clients connect directly** (`-ClientIpSource direct`, the default) - installs + URL Rewrite from the bundle and sets `X-Forwarded-For` from `REMOTE_ADDR`. + Overwriting the header is what stops a client spoofing its own. +- **A proxy sits in front** (`-ClientIpSource proxy`) - leaves the rule off. + Behind ARR or a load balancer `REMOTE_ADDR` is the *proxy*, so applying the + rule would discard the real client IP. + +An existing `web.config` is never overwritten - it is the one file on a server +that legitimately carries hand-edits. The installer reports what it found +instead. + ## Deployment methods Chosen in the wizard, and the bundle carries a SPA build for each because Vite diff --git a/deploy/windows/installer/build-installer.ps1 b/deploy/windows/installer/build-installer.ps1 new file mode 100644 index 0000000..d585dbd --- /dev/null +++ b/deploy/windows/installer/build-installer.ps1 @@ -0,0 +1,268 @@ +<# +.SYNOPSIS + Stage a lean per-site installer bundle on Windows, and verify its third-party + payload against bundle-lock.json. + +.DESCRIPTION + The Windows equivalent of build-installer.sh, for a work PC with no Bash. It + does the whole job natively - it does NOT shell out to build-site.sh - so the + only tools needed are Python, Node and (to compile) Inno Setup. + + What it does, in order: + + 1. Resolves the site's plugin closure from its profile. + 2. Builds the SPA twice, because Vite compiles the base path in and it + therefore cannot be chosen at install time: once for / (subpath + deployment) and once for / (its own site). + 3. Stages the backend tree: core, the chosen plugins only, and the runtime + files a deployable tree needs. + 4. Writes plugins.iss so the wizard's plugin page matches the payload. + 5. Copies the installer scripts from THIS directory. + 6. Verifies wheels\, python\ and the MSIs against bundle-lock.json, and + FAILS if the bundle is not exactly what the lock describes. + + Step 6 is the point of the script. A bundle that does not match its lock is + not shipped, and "the wheelhouse was missing" is caught here rather than by + an operator halfway through installing on a server with no network. + +.PARAMETER Profile + Path to the site profile (see deploy\site-profile.example.json). + +.PARAMETER RepoRoot + The repository. Defaults to four levels up from this script, which is correct + for a normal checkout. + +.PARAMETER SkipFrontend + Reuse the SPA builds already staged in the bundle. For iterating on the + installer itself, where two Vite builds per run is most of the wall clock. + Never use it for a bundle you intend to ship. + +.PARAMETER AllowUnlocked + Stage the bundle and report payload problems WITHOUT failing. For assembling + a bundle before its lock exists. A bundle built this way must not be shipped; + run refresh-bundle-lock.ps1, commit the lock, then build again without this. + +.EXAMPLE + .\build-installer.ps1 -Profile ..\..\site-profile.example.json + .\build-installer.ps1 -Profile C:\sites\wjf.json -SkipFrontend +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true)] [string] $Profile, + [string] $RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..\..')).Path, + [string] $BundleRoot = (Join-Path $PSScriptRoot 'bundle'), + [string] $SubpathAlias = 'shopdb', + [switch] $SkipFrontend, + [switch] $AllowUnlocked +) + +$ErrorActionPreference = 'Stop' +. (Join-Path $PSScriptRoot 'bundle-lock.ps1') + +function Say { param($m, $c = 'Gray') Write-Host $m -ForegroundColor $c } +function Step { param($m) Write-Host ''; Write-Host "==> $m" -ForegroundColor Cyan } +function Die { param($m, $fix = '') Write-Host ''; Write-Host " $m" -ForegroundColor Red + if ($fix) { Write-Host " $fix" -ForegroundColor Yellow }; exit 1 } + +function Invoke-Tool { + # Runs a build tool and stops on a non-zero exit. npm writes progress to + # stderr on SUCCESS, so stderr alone must never be treated as failure. + param([string] $Exe, [string[]] $Arguments, [string] $WorkDir, [string] $What) + Push-Location $WorkDir + try { + $prev = $ErrorActionPreference + $ErrorActionPreference = 'Continue' + & $Exe @Arguments 2>&1 | ForEach-Object { Say " $_" 'DarkGray' } + $code = $LASTEXITCODE + $ErrorActionPreference = $prev + if ($code -ne 0) { Die "$What failed (exit $code)" } + } finally { Pop-Location } +} + +if (-not (Test-Path $Profile)) { Die "profile not found: $Profile" } +if (-not (Test-Path $RepoRoot)) { Die "repo not found: $RepoRoot" } +$Profile = (Resolve-Path $Profile).Path +$RepoRoot = (Resolve-Path $RepoRoot).Path +$AppOut = Join-Path $BundleRoot 'app' + +$python = (Get-Command python -ErrorAction SilentlyContinue) +if (-not $python) { $python = Get-Command py -ErrorAction SilentlyContinue } +if (-not $python) { Die 'Python not found on PATH' 'Install Python 3.14 and re-open the shell.' } +$npm = Get-Command npm.cmd -ErrorAction SilentlyContinue +if (-not $npm -and -not $SkipFrontend) { Die 'npm not found on PATH' 'Install Node, or pass -SkipFrontend to reuse the staged SPA.' } + +Say '' +Say " repo : $RepoRoot" +Say " profile : $Profile" +Say " bundle : $BundleRoot" + +# --- 1. plugin closure ------------------------------------------------------ +# Same resolver the Linux builder uses, so both produce the same set from one +# profile instead of two implementations of the closure rules. +Step 'Resolving plugin closure' +$closure = (& $python.Source (Join-Path $RepoRoot 'scripts\resolve_plugin_closure.py') $Profile $RepoRoot) +if ($LASTEXITCODE -ne 0 -or -not $closure) { Die 'could not resolve the plugin closure from the profile' } +$closure = $closure.Trim() +Say " $closure" 'White' + +# --- 2. frontend ------------------------------------------------------------ +$frontend = Join-Path $RepoRoot 'frontend' +$subStaged = Join-Path $BundleRoot 'spa-subpath' +$rootStaged= Join-Path $BundleRoot 'spa-root' + +if ($SkipFrontend) { + if (-not (Test-Path $subStaged) -or -not (Test-Path $rootStaged)) { + Die '-SkipFrontend was passed but no SPA build is staged' 'Run once without it.' + } + Say '' + Say ' Reusing the staged SPA builds (-SkipFrontend). NOT shippable if the frontend changed.' 'Yellow' +} else { + Step "Building the SPA for /$SubpathAlias/" + $env:SITE_PLUGINS = $closure + $env:VITE_BASE_PATH = "/$SubpathAlias/" + try { Invoke-Tool $npm.Source @('run','build','--silent') $frontend 'subpath frontend build' } + finally { Remove-Item Env:\VITE_BASE_PATH -ErrorAction SilentlyContinue } + Remove-Item $subStaged -Recurse -Force -ErrorAction SilentlyContinue + Copy-Item (Join-Path $frontend 'dist') $subStaged -Recurse -Force + # The alias is fixed at BUILD time; the installer reads this and refuses to + # publish under a different one rather than serving a page that cannot load + # its own assets. + Set-Content -Path (Join-Path $subStaged '.alias') -Value $SubpathAlias -Encoding ASCII + + # Root build LAST, so frontend\dist is left in the state a developer expects. + Step 'Building the SPA for /' + try { Invoke-Tool $npm.Source @('run','build','--silent') $frontend 'root frontend build' } + finally { Remove-Item Env:\SITE_PLUGINS -ErrorAction SilentlyContinue } + Remove-Item $rootStaged -Recurse -Force -ErrorAction SilentlyContinue + Copy-Item (Join-Path $frontend 'dist') $rootStaged -Recurse -Force +} + +# --- 3. backend tree -------------------------------------------------------- +Step "Staging the application tree" +Remove-Item $AppOut -Recurse -Force -ErrorAction SilentlyContinue +New-Item -ItemType Directory -Path (Join-Path $AppOut 'plugins') -Force | Out-Null + +Copy-Item (Join-Path $RepoRoot 'shopdb') $AppOut -Recurse -Force +foreach ($name in $closure.Split(',')) { + $src = Join-Path $RepoRoot ('plugins\' + $name.Trim()) + if (-not (Test-Path $src)) { Die "plugin in the closure is not on disk: $name" } + Copy-Item $src (Join-Path $AppOut 'plugins') -Recurse -Force +} + +# Runtime files a deployable tree needs beyond the Python packages. Without +# these the tree imports but cannot be run or migrated. +foreach ($f in @('wsgi.py', 'requirements.txt')) { + Copy-Item (Join-Path $RepoRoot $f) $AppOut -Force +} +Copy-Item (Join-Path $RepoRoot 'migrations') $AppOut -Recurse -Force +# ONLY web.config, not all of deploy\. The bundle is staged INSIDE deploy\, so +# copying the whole tree would recurse into its own output, and the rest of +# deploy\ is installer source that has no business on an application server. +# shopdb-install.ps1 reads it from exactly this path. +$cfgSrc = Join-Path $RepoRoot 'deploy\windows\web.config' +if (Test-Path $cfgSrc) { + New-Item -ItemType Directory -Path (Join-Path $AppOut 'deploy\windows') -Force | Out-Null + Copy-Item $cfgSrc (Join-Path $AppOut 'deploy\windows') -Force +} + +# Stage the profile INTO the tree: `flask plugin apply-profile` at provisioning +# reads the same profile the tree was staged from, so the installed plugin set +# and the shipped plugin code cannot drift. +Copy-Item $Profile (Join-Path $AppOut 'site-profile.json') -Force + +# The installer expects frontend\dist and frontend\dist-subpath. +$feOut = Join-Path $AppOut 'frontend' +New-Item -ItemType Directory -Path $feOut -Force | Out-Null +Copy-Item $rootStaged (Join-Path $feOut 'dist') -Recurse -Force +Copy-Item $subStaged (Join-Path $feOut 'dist-subpath') -Recurse -Force + +Get-ChildItem $AppOut -Recurse -Directory -Filter '__pycache__' -ErrorAction SilentlyContinue | + Remove-Item -Recurse -Force -ErrorAction SilentlyContinue +Get-ChildItem $AppOut -Recurse -File -Filter '*.pyc' -ErrorAction SilentlyContinue | + Remove-Item -Force -ErrorAction SilentlyContinue + +# --- 4. plugins.iss --------------------------------------------------------- +Step 'Writing plugins.iss' +$shipped = (Get-ChildItem (Join-Path $AppOut 'plugins') -Directory | Select-Object -ExpandProperty Name) -join ',' +$aliasBuilt = '' +$aliasFile = Join-Path $feOut 'dist-subpath\.alias' +if (Test-Path $aliasFile) { $aliasBuilt = (Get-Content $aliasFile -TotalCount 1).Trim() } +@" +; GENERATED by build-installer.ps1 - do not edit. +; The plugins present in bundle\app\plugins at build time. +#define AvailablePlugins "$shipped" +; The alias the subpath SPA was built for, or empty if this bundle has no +; subpath build - in which case the wizard must not offer that option. +#define SubpathAlias "$aliasBuilt" +"@ | Set-Content -Path (Join-Path $PSScriptRoot 'plugins.iss') -Encoding ASCII +Say " $shipped" 'White' + +# --- 5. installer scripts --------------------------------------------------- +# From THIS directory, which is the reviewed copy under version control. They +# used to be copied from a downloads folder, so the logic that shipped was not +# the logic that was committed and the build only worked on one machine. +Step 'Copying installer scripts' +foreach ($f in @('shopdb-install.ps1', 'shopdb-preflight.ps1', 'bundle-lock.ps1')) { + $src = Join-Path $PSScriptRoot $f + if (-not (Test-Path $src)) { Die "installer script missing from the repo: $f" } + Copy-Item $src $BundleRoot -Force + Say " $f" +} + +# --- 6. payload verification ------------------------------------------------ +Step 'Verifying the third-party payload against bundle-lock.json' +$lockPath = Join-Path $PSScriptRoot 'bundle-lock.json' +$lock = Read-BundleLock $lockPath +if (-not $lock) { + $msg = "no bundle-lock.json at $lockPath" + if ($AllowUnlocked) { Say " $msg - continuing because -AllowUnlocked was passed" 'Yellow' } + else { + Die $msg @' +Add the wheels and installers to the bundle, then: + .\refresh-bundle-lock.ps1 (review the list) + .\refresh-bundle-lock.ps1 -Yes (write it) +and commit bundle-lock.json. Pass -AllowUnlocked to stage a bundle without one - +it must not be shipped. +'@ + } +} else { + $problems = Test-BundleLock -BundleRoot $BundleRoot -Lock $lock + if ($problems.Count -eq 0) { + Say (" payload matches the lock ({0}, {1})" -f ` + (Get-JsonProperty $lock 'pythontag' 'unknown'), (Get-JsonProperty $lock 'platform' 'unknown')) 'Green' + # Ships WITH the bundle: the installer re-checks the payload on the + # target server before running any of it, so tampering between build and + # install is caught too. + Copy-Item $lockPath $BundleRoot -Force + } else { + Say '' + foreach ($p in $problems) { Say " $p" 'Red' } + Say '' + if ($AllowUnlocked) { + Say ' -AllowUnlocked: continuing anyway. DO NOT SHIP this bundle.' 'Yellow' + } else { + Die ("{0} payload problem(s) - the bundle is not what the lock describes" -f $problems.Count) @' +Either the payload is wrong (fix the bundle) or it changed on purpose (run +refresh-bundle-lock.ps1, read the diff, and commit the new lock). +'@ + } + } +} + +# --- summary ---------------------------------------------------------------- +Write-Host '' +Say " Bundle staged at: $BundleRoot" 'White' +foreach ($d in @('app', 'wheels', 'python', 'httpplatformhandler', 'urlrewrite', 'mysql')) { + $p = Join-Path $BundleRoot $d + if (Test-Path $p) { + $mb = ((Get-ChildItem $p -Recurse -File | Measure-Object Length -Sum).Sum / 1MB) + Say (" {0,-22} {1,8:N1} MB" -f $d, $mb) + } else { + Say (" {0,-22} {1}" -f $d, 'absent') 'DarkGray' + } +} +Write-Host '' +Say ' Compile with: iscc ShopDBFlask.iss' 'White' +Say ' (Inno Setup 6.6.0 or newer - the wizard uses the windows11 custom style.)' 'DarkGray' +Write-Host '' diff --git a/deploy/windows/installer/build-installer.sh b/deploy/windows/installer/build-installer.sh index 115b09e..f20a0d9 100755 --- a/deploy/windows/installer/build-installer.sh +++ b/deploy/windows/installer/build-installer.sh @@ -59,26 +59,71 @@ cat > "$HERE/plugins.iss" < Copying installer scripts" mkdir -p "$BUNDLE" -cp "$HOME/Downloads/shopdb-install.ps1" "$HOME/Downloads/shopdb-preflight.ps1" "$BUNDLE/" +for f in shopdb-install.ps1 shopdb-preflight.ps1 bundle-lock.ps1; do + [ -f "$HERE/$f" ] || { echo "installer script missing from the repo: $f"; exit 1; } + cp "$HERE/$f" "$BUNDLE/" +done echo "" echo "Bundle staged at: $BUNDLE" -for d in app wheels python httpplatformhandler mysql; do +for d in app wheels python httpplatformhandler urlrewrite mysql; do if [ -d "$BUNDLE/$d" ]; then printf ' %-20s %s\n' "$d" "$(du -sh "$BUNDLE/$d" | cut -f1)" else - printf ' %-20s MISSING\n' "$d" + printf ' %-20s absent\n' "$d" fi done echo "" echo " plugins shipped: $(ls "$BUNDLE/app/plugins" 2>/dev/null | tr '\n' ' ')" + +# --- payload verification --------------------------------------------------- +# The third-party payload is the part git does not record: the wheels, the Python +# installer and the MSIs that run as SYSTEM on the target server. It must be +# EXACTLY what bundle-lock.json describes - no missing file, no stale extra wheel +# left over from a previous build, no changed content - or this is not a bundle +# anyone reviewed. Previously a missing wheelhouse printed MISSING and the script +# still exited 0, so an empty bundle compiled into a shippable installer and the +# failure surfaced on an air-gapped server with no way to fix it. +# +# ALLOW_UNLOCKED=1 downgrades this to a warning, for assembling a bundle before +# its lock exists. A bundle built that way must not be shipped. echo "" -echo "Missing pieces must be added by hand before compiling:" -echo " wheels\\ built ON Windows (cp314 win_amd64), ~47 wheels" -echo " python\\ python-3.14.6-amd64.exe" -echo " httpplatformhandler\\ httpPlatformHandler_amd64.msi" -echo " mysql\\ mysql-8.0.46-winx64.msi (bundled-database option only)" +echo "==> Verifying the third-party payload against bundle-lock.json" +if python3 "$HERE/verify_bundle_lock.py" "$BUNDLE" "$HERE/bundle-lock.json"; then + echo " payload matches the lock" + # Ships WITH the bundle: the installer re-checks the payload on the target + # server before running any of it, so tampering between build and install is + # caught too. + cp "$HERE/bundle-lock.json" "$BUNDLE/" +elif [ "${ALLOW_UNLOCKED:-0}" = "1" ]; then + echo "" + echo " ALLOW_UNLOCKED=1: continuing anyway. DO NOT SHIP this bundle." +else + echo "" + echo " The bundle is not what the lock describes." + echo "" + echo " Add the missing pieces by hand:" + echo " wheels/ pip download -r requirements.txt --only-binary=:all: \\" + echo " --platform win_amd64 --python-version 314 \\" + echo " --implementation cp --abi cp314 -d wheels" + echo " python/ python-3.14.x-amd64.exe" + echo " httpplatformhandler/ httpPlatformHandler_amd64.msi" + echo " urlrewrite/ rewrite_amd64.msi (client-IP rule; see README)" + echo " mysql/ mysql-8.0.x-winx64.msi (bundled-database option only)" + echo "" + echo " If the payload changed ON PURPOSE, regenerate and COMMIT the lock:" + echo " pwsh ./refresh-bundle-lock.ps1 # review the diff" + echo " pwsh ./refresh-bundle-lock.ps1 -Yes # write it" + echo "" + echo " To stage a bundle before its lock exists: ALLOW_UNLOCKED=1 $0 ..." + exit 1 +fi + echo "" echo "Then compile on Windows: iscc ShopDBFlask.iss" +echo "(Inno Setup 6.6.0 or newer - the wizard uses the windows11 custom style.)" diff --git a/deploy/windows/installer/bundle-lock.ps1 b/deploy/windows/installer/bundle-lock.ps1 new file mode 100644 index 0000000..6695fcc --- /dev/null +++ b/deploy/windows/installer/bundle-lock.ps1 @@ -0,0 +1,192 @@ +<# +.SYNOPSIS + Hash manifest for the installer's third-party payload: create it, and check a + bundle against it. + +.DESCRIPTION + Dot-source this. It defines two functions and runs nothing on its own: + + New-BundleLock hash a staged bundle and return the lock object + Test-BundleLock compare a staged bundle against a lock, return problems + + WHAT IT COVERS, and why pip's own hash checking is not enough. + + requirements.txt carries a sha256 for every wheel, so pip refuses an artifact + upstream did not publish. Three gaps remain, and all three are what actually + goes wrong with a hand-assembled offline bundle: + + 1. pip lists EVERY artifact of a pinned version - cffi 2.1.0 alone has 100 + hashes. It proves the wheel is genuine, not that it is the wheel this + bundle was built and tested with. + 2. pip ignores extra files in the wheelhouse. A stale wheel left behind by + a previous build sits there unnoticed until a resolve picks it up. + 3. pip says nothing about the rest of the payload - the Python installer, + the HttpPlatformHandler MSI, URL Rewrite, MySQL. Those are executables + that run as SYSTEM on the target server and were, until this file, the + only unverified thing the installer would run. + + So the lock records an exact file set with a sha256 and a byte size each, and + verification is SET EQUALITY: a missing file, an unexpected extra file, or a + changed file all fail. Nothing is skipped and nothing is "close enough". + + The app tree is deliberately NOT covered. It is built from the repository on + every run and changes with every commit; hashing it would make the lock churn + constantly and train everyone to regenerate it without reading it. Git is the + record for the app tree. This file is the record for everything that comes + from outside the repository. + +.NOTES + Stock Windows PowerShell 5.1, and also runs under pwsh on Linux so the Bash + builder can call the same checker. +#> + +# Payload directories under the bundle root. 'required' means the installer +# cannot work without it; the optional ones are per-deployment choices, and an +# absent optional directory is fine. A PRESENT directory is always checked in +# full, optional or not. +$script:BundlePayloads = @( + @{ Name = 'wheels'; Required = $true; What = 'Python wheels for the offline install' } + @{ Name = 'python'; Required = $true; What = 'the Python installer' } + @{ Name = 'httpplatformhandler'; Required = $true; What = 'the IIS module that launches waitress' } + @{ Name = 'urlrewrite'; Required = $false; What = 'IIS URL Rewrite, for the client-IP rule' } + @{ Name = 'mysql'; Required = $false; What = 'MySQL, for the bundled-database option' } +) + +function Get-JsonProperty { + # shopdb-install.ps1 runs under Set-StrictMode 2.0, where reading a property + # that does not exist on a PSCustomObject THROWS instead of returning $null. + # A truncated or hand-edited bundle-lock.json would therefore blow up with + # "Property 'payloads' cannot be found" rather than saying what is wrong with + # the lock. Every read of parsed JSON goes through here. + param($Object, [string] $Name, $Default = $null) + if ($null -eq $Object) { return $Default } + $prop = $Object.PSObject.Properties[$Name] + if ($null -eq $prop) { return $Default } + return $prop.Value +} + +function Get-FileDigest { + param([string] $Path) + $sha = [System.Security.Cryptography.SHA256]::Create() + $stream = [System.IO.File]::OpenRead($Path) + try { return (-join ($sha.ComputeHash($stream) | ForEach-Object { $_.ToString('x2') })) } + finally { $stream.Dispose(); $sha.Dispose() } +} + +function Get-PayloadFiles { + # Every file in the directory, keyed by its path RELATIVE to that directory + # with forward slashes, so a lock generated on Windows reads the same from + # the Bash builder. + param([string] $Dir) + $out = @{} + if (-not (Test-Path $Dir)) { return $out } + $root = (Resolve-Path $Dir).Path.TrimEnd('\', '/') + foreach ($f in (Get-ChildItem $Dir -Recurse -File)) { + $rel = $f.FullName.Substring($root.Length).TrimStart('\', '/').Replace('\', '/') + $out[$rel] = @{ sha256 = (Get-FileDigest $f.FullName); size = $f.Length } + } + return $out +} + +function New-BundleLock { + <# + Hash a staged bundle. The caller writes the result to bundle-lock.json; + this returns the object so a caller can diff it against the committed lock + before overwriting anything. + #> + param( + [Parameter(Mandatory = $true)] [string] $BundleRoot, + [string] $PythonTag = 'cp314', + [string] $Platform = 'win_amd64' + ) + $payloads = @{} + foreach ($p in $script:BundlePayloads) { + $dir = Join-Path $BundleRoot $p.Name + if (-not (Test-Path $dir)) { + if ($p.Required) { throw ("payload directory is missing: {0} ({1})" -f $p.Name, $p.What) } + continue + } + $files = Get-PayloadFiles $dir + if ($files.Count -eq 0 -and $p.Required) { + throw ("payload directory is empty: {0} ({1})" -f $p.Name, $p.What) + } + $payloads[$p.Name] = @{ required = $p.Required; files = $files } + } + return [ordered]@{ + schema = 1 + generated = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ') + pythontag = $PythonTag + platform = $Platform + note = 'Exact third-party payload of the installer bundle. Regenerate with refresh-bundle-lock.ps1 and COMMIT the change as a reviewed dependency bump.' + payloads = $payloads + } +} + +function Test-BundleLock { + <# + Compare a staged bundle against a lock. Returns an array of problem + strings - EMPTY means the bundle is exactly what the lock describes. + + Returning problems rather than throwing is deliberate: an operator fixing a + wheelhouse wants the whole list at once, not one failure per rebuild. + #> + param( + [Parameter(Mandatory = $true)] [string] $BundleRoot, + [Parameter(Mandatory = $true)] $Lock + ) + $problems = @() + $payloads = Get-JsonProperty $Lock 'payloads' + if ($null -eq $payloads) { return @('bundle-lock.json has no "payloads" section') } + + foreach ($p in $script:BundlePayloads) { + $name = $p.Name + $dir = Join-Path $BundleRoot $name + $present = Test-Path $dir + $entry = Get-JsonProperty $payloads $name + $locked = ($null -ne $entry) + + if (-not $locked) { + # Not in the lock at all. An unlocked directory that exists is a + # payload nobody reviewed, which is exactly what this is here to stop. + if ($present) { $problems += "$name/ is present but is not in bundle-lock.json - regenerate the lock" } + elseif ($p.Required) { $problems += "$name/ is required but is in neither the bundle nor the lock" } + continue + } + if (-not $present) { + if ($p.Required -or (Get-JsonProperty $entry 'required' $false)) { + $problems += "$name/ is in the lock but missing from the bundle ($($p.What))" + } + continue + } + + $expected = @{} + $lockedFiles = Get-JsonProperty $entry 'files' + if ($null -ne $lockedFiles) { + foreach ($prop in $lockedFiles.PSObject.Properties) { $expected[$prop.Name] = $prop.Value } + } + $actual = Get-PayloadFiles $dir + + # Sorted so the report reads the same way twice, and so it matches the + # order verify_bundle_lock.py produces. + foreach ($rel in ($expected.Keys | Sort-Object)) { + if (-not $actual.ContainsKey($rel)) { $problems += "$name/$rel is in the lock but missing from the bundle"; continue } + if ($actual[$rel].sha256 -ne $expected[$rel].sha256) { + $problems += "$name/$rel does NOT match the lock (expected sha256 $($expected[$rel].sha256.Substring(0,12))..., got $($actual[$rel].sha256.Substring(0,12))...)" + } elseif ([int64] $actual[$rel].size -ne [int64] $expected[$rel].size) { + # Cannot happen for a matching sha256, so it means the lock itself + # was hand-edited. Say so rather than passing it. + $problems += "$name/$rel size disagrees with the lock - the lock has been edited by hand" + } + } + foreach ($rel in ($actual.Keys | Sort-Object)) { + if (-not $expected.ContainsKey($rel)) { $problems += "$name/$rel is in the bundle but NOT in the lock (unexpected extra file)" } + } + } + return $problems +} + +function Read-BundleLock { + param([Parameter(Mandatory = $true)] [string] $Path) + if (-not (Test-Path $Path)) { return $null } + return (Get-Content $Path -Raw | ConvertFrom-Json) +} diff --git a/deploy/windows/installer/make-branding.py b/deploy/windows/installer/make-branding.py index 20caf30..e7637a2 100644 --- a/deploy/windows/installer/make-branding.py +++ b/deploy/windows/installer/make-branding.py @@ -150,7 +150,7 @@ def main(): for s, name in [(55, "wizard-small.bmp"), (64, "wizard-small@125.bmp"), (138, "wizard-small@250.bmp")]: small(s).save(OUT / name, "BMP"); made.append((name, f"{s}x{s}")) - icon(OUT / "shopdb.ico"); made.append(("shopdb.ico", "multi-res")) + icon(OUT / "shopdb.ico"); made.append(("shopdb.ico", "multi-size")) for name, dims in made: print(f" {name:<26} {dims:<10} {(OUT / name).stat().st_size // 1024} KB") diff --git a/deploy/windows/installer/refresh-bundle-lock.ps1 b/deploy/windows/installer/refresh-bundle-lock.ps1 new file mode 100644 index 0000000..95849c8 --- /dev/null +++ b/deploy/windows/installer/refresh-bundle-lock.ps1 @@ -0,0 +1,110 @@ +<# +.SYNOPSIS + Regenerate bundle-lock.json from the staged bundle, after showing what would + change. + +.DESCRIPTION + Run this on the machine that assembled the wheelhouse, once the bundle holds + the payload you intend to ship: + + bundle\wheels\ the wheels, built with the matching Python + bundle\python\ the Python installer + bundle\httpplatformhandler\ the IIS module MSI + bundle\urlrewrite\ URL Rewrite MSI (optional) + bundle\mysql\ MySQL MSI (optional) + + Then COMMIT the resulting bundle-lock.json. That commit is the review: it is + the only place a change to what runs as SYSTEM on a customer's server becomes + visible to a human. A lock regenerated and committed without reading the diff + provides nothing, so this refuses to overwrite an existing lock until you + have seen the change and passed -Yes. + +.EXAMPLE + .\refresh-bundle-lock.ps1 # show the diff, write nothing + .\refresh-bundle-lock.ps1 -Yes # write it + +.NOTES + Building the wheelhouse itself no longer requires Windows. From any machine: + pip download -r requirements.txt -d wheels --only-binary=:all: ` + --platform win_amd64 --python-version 314 --implementation cp --abi cp314 + Do it wherever you like; this script records what came out. +#> + +[CmdletBinding()] +param( + [string] $BundleRoot = (Join-Path $PSScriptRoot 'bundle'), + [string] $LockPath = (Join-Path $PSScriptRoot 'bundle-lock.json'), + [string] $PythonTag = 'cp314', + [string] $Platform = 'win_amd64', + [switch] $Yes +) + +$ErrorActionPreference = 'Stop' +. (Join-Path $PSScriptRoot 'bundle-lock.ps1') + +function Say { param($m, $c = 'Gray') Write-Host $m -ForegroundColor $c } + +if (-not (Test-Path $BundleRoot)) { + Say "bundle not found: $BundleRoot" 'Red' + Say 'Stage it first with build-installer.ps1 (or build-installer.sh), then add' 'Yellow' + Say 'the wheels and installers by hand.' 'Yellow' + exit 1 +} + +Say '' +Say " Hashing $BundleRoot" 'Cyan' +$new = New-BundleLock -BundleRoot $BundleRoot -PythonTag $PythonTag -Platform $Platform + +foreach ($name in $new.payloads.Keys) { + Say (" {0,-22} {1,4} files" -f $name, $new.payloads[$name].files.Count) +} + +$old = Read-BundleLock $LockPath +if (-not $old) { + Say '' + Say ' No existing lock - this will be the first one.' 'Yellow' +} else { + # Diff by file, per payload, so the operator sees exactly which artifacts + # changed rather than "the lock is different". + Say '' + Say ' Changes against the committed lock:' 'Cyan' + $changes = 0 + foreach ($name in $new.payloads.Keys) { + $oldFiles = @{} + if ($old.payloads.PSObject.Properties.Name -contains $name) { + foreach ($p in $old.payloads.$name.files.PSObject.Properties) { $oldFiles[$p.Name] = $p.Value.sha256 } + } + $newFiles = $new.payloads[$name].files + foreach ($rel in ($newFiles.Keys | Sort-Object)) { + if (-not $oldFiles.ContainsKey($rel)) { Say " + $name/$rel" 'Green'; $changes++ } + elseif ($oldFiles[$rel] -ne $newFiles[$rel].sha256) { Say " ~ $name/$rel (content changed)" 'Yellow'; $changes++ } + } + foreach ($rel in ($oldFiles.Keys | Sort-Object)) { + if (-not $newFiles.ContainsKey($rel)) { Say " - $name/$rel" 'Red'; $changes++ } + } + } + foreach ($p in $old.payloads.PSObject.Properties.Name) { + if (-not $new.payloads.Contains($p)) { Say " - $p/ (whole payload gone)" 'Red'; $changes++ } + } + if ($changes -eq 0) { + Say ' none - the bundle already matches the lock' 'Green' + exit 0 + } + Say '' + Say (" {0} change(s)." -f $changes) 'White' +} + +if (-not $Yes) { + Say '' + Say ' Nothing written. Read the list above, then re-run with -Yes.' 'Yellow' + Say ' Commit the resulting bundle-lock.json - that commit IS the review.' 'Yellow' + exit 2 +} + +# ConvertTo-Json defaults to a depth of 2, which silently flattens the per-file +# entries into "System.Collections.Hashtable" strings and produces a lock that +# verifies against nothing. +$new | ConvertTo-Json -Depth 8 | Set-Content -Path $LockPath -Encoding UTF8 +Say '' +Say " Written: $LockPath" 'Green' +Say ' Commit it.' 'Green' diff --git a/deploy/windows/installer/verify_bundle_lock.py b/deploy/windows/installer/verify_bundle_lock.py new file mode 100644 index 0000000..2c11c57 --- /dev/null +++ b/deploy/windows/installer/verify_bundle_lock.py @@ -0,0 +1,119 @@ +#!/usr/bin/env python3 +"""Check a staged installer bundle against bundle-lock.json. + +Prints one line per problem and exits non-zero if there are any. Exits 0 only +when the bundle's third-party payload is EXACTLY what the lock describes: no +missing file, no unexpected extra file, no changed content. + +Why this exists alongside bundle-lock.ps1, which does the same job: + + - bundle-lock.ps1 is canonical. It runs at INSTALL time on the target server, + where PowerShell is the only thing guaranteed to be present - Python is not + installed until stage 2, and verifying the payload after running part of it + would defeat the purpose. + - This file lets the Linux builder (build-installer.sh) do the same check + without adding pwsh as a build dependency. + +The two are kept honest by tests/test_bundle_lock.py, which runs BOTH against +the same fixtures and fails if they disagree. + +Usage: verify_bundle_lock.py +""" +import hashlib +import json +import os +import sys + +# Must match $script:BundlePayloads in bundle-lock.ps1. +PAYLOADS = [ + ('wheels', True, 'Python wheels for the offline install'), + ('python', True, 'the Python installer'), + ('httpplatformhandler', True, 'the IIS module that launches waitress'), + ('urlrewrite', False, 'IIS URL Rewrite, for the client-IP rule'), + ('mysql', False, 'MySQL, for the bundled-database option'), +] + + +def digest(path): + sha = hashlib.sha256() + with open(path, 'rb') as fh: + for chunk in iter(lambda: fh.read(1024 * 1024), b''): + sha.update(chunk) + return sha.hexdigest() + + +def payload_files(directory): + """Every file under the directory, keyed by forward-slashed relative path.""" + found = {} + if not os.path.isdir(directory): + return found + for root, _dirs, files in os.walk(directory): + for name in files: + full = os.path.join(root, name) + rel = os.path.relpath(full, directory).replace(os.sep, '/') + found[rel] = {'sha256': digest(full), 'size': os.path.getsize(full)} + return found + + +def verify(bundle_root, lock): + problems = [] + locked = lock.get('payloads') + if not locked: + return ['bundle-lock.json has no "payloads" section'] + + for name, required, what in PAYLOADS: + directory = os.path.join(bundle_root, name) + present = os.path.isdir(directory) + if name not in locked: + if present: + problems.append( + '%s/ is present but is not in bundle-lock.json - regenerate the lock' % name) + elif required: + problems.append( + '%s/ is required but is in neither the bundle nor the lock' % name) + continue + if not present: + if required or locked[name].get('required'): + problems.append('%s/ is in the lock but missing from the bundle (%s)' % (name, what)) + continue + + expected = locked[name].get('files', {}) + actual = payload_files(directory) + for rel, want in sorted(expected.items()): + got = actual.get(rel) + if got is None: + problems.append('%s/%s is in the lock but missing from the bundle' % (name, rel)) + elif got['sha256'] != want['sha256']: + problems.append( + '%s/%s does NOT match the lock (expected sha256 %s..., got %s...)' + % (name, rel, want['sha256'][:12], got['sha256'][:12])) + elif int(got['size']) != int(want['size']): + # Impossible for a matching sha256, so the lock was hand-edited. + problems.append( + '%s/%s size disagrees with the lock - the lock has been edited by hand' + % (name, rel)) + for rel in sorted(actual): + if rel not in expected: + problems.append( + '%s/%s is in the bundle but NOT in the lock (unexpected extra file)' + % (name, rel)) + return problems + + +def main(): + if len(sys.argv) != 3: + sys.exit('usage: verify_bundle_lock.py ') + bundle_root, lock_path = sys.argv[1], sys.argv[2] + if not os.path.exists(lock_path): + print('no bundle-lock.json at %s' % lock_path) + return 1 + with open(lock_path) as fh: + lock = json.load(fh) + problems = verify(bundle_root, lock) + for problem in problems: + print(problem) + return 1 if problems else 0 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/tests/test_bundle_lock.py b/tests/test_bundle_lock.py new file mode 100644 index 0000000..6b45f3a --- /dev/null +++ b/tests/test_bundle_lock.py @@ -0,0 +1,199 @@ +"""The installer bundle lock: fail-closed behaviour, and parity between the two +implementations that read it. + +There are two verifiers on purpose - bundle-lock.ps1 runs at install time on a +server where PowerShell is the only thing present, verify_bundle_lock.py runs in +the Linux builder without adding pwsh as a build dependency. Two implementations +of one rule drift. These tests run BOTH against the same fixtures and fail if +they disagree, so the drift shows up here rather than as a bundle that one of +them waves through. + +The PowerShell half is skipped where pwsh is absent; the Python half always runs. +""" +import json +import re +import shutil +import subprocess +import sys +from pathlib import Path + +import pytest + +INSTALLER = Path(__file__).resolve().parents[1] / 'deploy' / 'windows' / 'installer' +VERIFY_PY = INSTALLER / 'verify_bundle_lock.py' +LOCK_PS1 = INSTALLER / 'bundle-lock.ps1' +PWSH = shutil.which('pwsh') or shutil.which('powershell') + +pytestmark = pytest.mark.skipif(not VERIFY_PY.exists(), reason='installer not in this tree') + + +def build_bundle(root): + """A minimal bundle holding every REQUIRED payload directory.""" + for name, files in ( + ('wheels', {'alembic-1.18.4-py3-none-any.whl': b'wheel-a', + 'cffi-2.1.0-cp314-cp314-win_amd64.whl': b'wheel-b'}), + ('python', {'python-3.14.6-amd64.exe': b'py'}), + ('httpplatformhandler', {'httpPlatformHandler_amd64.msi': b'hph'}), + ): + directory = root / name + directory.mkdir(parents=True) + for filename, content in files.items(): + (directory / filename).write_bytes(content) + return root + + +def write_lock(bundle, lock_path): + """Generate the lock the same way refresh-bundle-lock.ps1 does.""" + sys.path.insert(0, str(INSTALLER)) + import verify_bundle_lock as verifier + + payloads = {} + for name, required, _what in verifier.PAYLOADS: + directory = bundle / name + if not directory.is_dir(): + continue + payloads[name] = {'required': required, 'files': verifier.payload_files(str(directory))} + lock_path.write_text(json.dumps({ + 'schema': 1, 'generated': '2026-08-03T00:00:00Z', + 'pythontag': 'cp314', 'platform': 'win_amd64', 'payloads': payloads, + })) + + +def check_python(bundle, lock_path): + result = subprocess.run( + [sys.executable, str(VERIFY_PY), str(bundle), str(lock_path)], + capture_output=True, text=True) + problems = [line for line in result.stdout.splitlines() if line.strip()] + # Exit status and output must agree, or a caller that checks only one of them + # gets a different answer from a caller that checks the other. + assert (result.returncode != 0) == bool(problems), result.stdout + return problems + + +def check_powershell(bundle, lock_path): + script = ( + ". '%s'; $p = Test-BundleLock -BundleRoot '%s' -Lock (Read-BundleLock '%s'); " + "if ($p) { $p -join \"`n\" }" % (LOCK_PS1, bundle, lock_path)) + result = subprocess.run([PWSH, '-NoProfile', '-Command', script], + capture_output=True, text=True) + assert result.returncode == 0, result.stderr + return [line for line in result.stdout.splitlines() if line.strip()] + + +def check_both(bundle, lock_path): + """Returns the Python verdict, asserting PowerShell reports the same set. + + Compared as sets: both sort their output, but PowerShell's Sort-Object is + culture-aware and Python's sorted() is ordinal, so the two can order the same + findings differently. What must never differ is WHICH problems are found. + """ + problems = check_python(bundle, lock_path) + if PWSH and LOCK_PS1.exists(): + assert sorted(check_powershell(bundle, lock_path)) == sorted(problems), ( + 'verify_bundle_lock.py and bundle-lock.ps1 disagree') + return problems + + +@pytest.fixture +def locked(tmp_path): + bundle = build_bundle(tmp_path / 'bundle') + lock = tmp_path / 'bundle-lock.json' + write_lock(bundle, lock) + return bundle, lock + + +def test_untouched_bundle_passes(locked): + bundle, lock = locked + assert check_both(bundle, lock) == [] + + +def test_extra_file_fails(locked): + """The gap pip's own hash checking leaves: a stale wheel nobody asked for.""" + bundle, lock = locked + (bundle / 'wheels' / 'stale-0.1-py3-none-any.whl').write_bytes(b'left over') + problems = check_both(bundle, lock) + assert any('stale-0.1' in p and 'NOT in the lock' in p for p in problems) + + +def test_altered_file_fails(locked): + bundle, lock = locked + (bundle / 'wheels' / 'cffi-2.1.0-cp314-cp314-win_amd64.whl').write_bytes(b'swapped') + problems = check_both(bundle, lock) + assert any('does NOT match the lock' in p for p in problems) + + +def test_missing_file_fails(locked): + bundle, lock = locked + (bundle / 'wheels' / 'alembic-1.18.4-py3-none-any.whl').unlink() + problems = check_both(bundle, lock) + assert any('missing from the bundle' in p for p in problems) + + +def test_missing_required_payload_fails(locked): + bundle, lock = locked + shutil.rmtree(bundle / 'python') + problems = check_both(bundle, lock) + assert any(p.startswith('python/') for p in problems) + + +def test_unlocked_payload_directory_fails(locked): + """An optional payload dropped in after the lock was made is still unreviewed.""" + bundle, lock = locked + (bundle / 'mysql').mkdir() + (bundle / 'mysql' / 'mysql-8.0.46-winx64.msi').write_bytes(b'msi') + problems = check_both(bundle, lock) + assert any('mysql/ is present but is not in bundle-lock.json' in p for p in problems) + + +def test_absent_optional_payload_is_fine(locked): + bundle, lock = locked + assert not (bundle / 'mysql').exists() + assert check_both(bundle, lock) == [] + + +def test_missing_lock_fails_closed(locked): + """No lock is a failure, not a pass. The build must not fall through to 'fine'.""" + bundle, lock = locked + lock.unlink() + result = subprocess.run([sys.executable, str(VERIFY_PY), str(bundle), str(lock)], + capture_output=True, text=True) + assert result.returncode != 0 + assert 'no bundle-lock.json' in result.stdout + + +def test_malformed_lock_reports_rather_than_crashing(locked): + """A truncated lock must produce a sentence, not a stack trace. + + shopdb-install.ps1 runs under Set-StrictMode 2.0, where reading a property + that is not there throws. Both verifiers have to survive a lock missing the + section they read first. + """ + bundle, lock = locked + lock.write_text(json.dumps({'schema': 1, 'pythontag': 'cp314'})) + problems = check_both(bundle, lock) + assert problems == ['bundle-lock.json has no "payloads" section'] + + +def test_lock_missing_files_section_is_not_a_silent_pass(locked): + """A payload entry with no file list must not read as 'nothing expected, fine'.""" + bundle, lock = locked + data = json.loads(lock.read_text()) + del data['payloads']['wheels']['files'] + lock.write_text(json.dumps(data)) + problems = check_both(bundle, lock) + assert problems, 'an entry with no file list let every wheel through' + assert all('wheels/' in p for p in problems) + + +def test_payload_list_matches_powershell(): + """Both files enumerate the payload directories. Same names, same required + flags, same order - a directory that is required in one and optional in the + other means the builder and the installer disagree about what may be absent. + """ + sys.path.insert(0, str(INSTALLER)) + import verify_bundle_lock as verifier + + declared = re.findall(r"Name\s*=\s*'([a-z]+)'\s*;\s*Required\s*=\s*\$(true|false)", + LOCK_PS1.read_text()) + assert declared, 'could not find $script:BundlePayloads in bundle-lock.ps1' + assert declared == [(name, str(required).lower()) for name, required, _ in verifier.PAYLOADS]