Files
shopdb-flask/deploy/windows/installer/bundle-lock.ps1
cproudlock 88af7fd9ce feat(installer): lock the third-party payload, and build on Windows without Bash
The bundle carries ~40 wheels, a Python installer and two MSIs. All of them run
as SYSTEM on the target server, and nothing verified any of them. A missing
wheelhouse printed MISSING and the script still exited 0, so an empty bundle
compiled into a shippable installer and the failure surfaced on an air-gapped
server with no way to fix it.

bundle-lock.json now records that payload exactly - sha256 and byte size per
file - and verification is set equality: a missing file, an unexpected extra
file, or changed content all fail. Both builders check it and refuse to produce
an unverified bundle; the lock ships inside the bundle and shopdb-install.ps1
re-checks it on the server before running any of it.

This is deliberately a layer above requirements.txt hashes. pip lists every
artifact of a pinned version (cffi 2.1.0 alone has 100 hashes), so it proves a
wheel is genuine, not that it is the wheel this bundle was built and tested
with; it ignores extra files in the wheelhouse; and it covers none of the
executables.

refresh-bundle-lock.ps1 regenerates the lock but refuses to overwrite one until
the operator has seen the diff, because the commit is the review - it is the
only place a change to what runs as SYSTEM becomes visible to a human.

build-installer.ps1 is the whole build natively on Windows, so a work PC needs
no Bash. It shares the plugin closure resolver with build-site.sh.

Both builders now copy the installer scripts from the repository. They were
copied from a downloads folder, so the logic that shipped was not the logic that
was committed and the build worked on exactly one machine.

Two verifiers exist because PowerShell is the only thing guaranteed present on
the target server, while the Linux builder should not need pwsh.
tests/test_bundle_lock.py runs both against the same fixtures and fails if they
disagree.
2026-08-03 11:17:45 -04:00

193 lines
8.7 KiB
PowerShell

<#
.SYNOPSIS
Hash manifest for the installer's third-party payload: create it, and check a
bundle against it.
.DESCRIPTION
Dot-source this. It defines two functions and runs nothing on its own:
New-BundleLock hash a staged bundle and return the lock object
Test-BundleLock compare a staged bundle against a lock, return problems
WHAT IT COVERS, and why pip's own hash checking is not enough.
requirements.txt carries a sha256 for every wheel, so pip refuses an artifact
upstream did not publish. Three gaps remain, and all three are what actually
goes wrong with a hand-assembled offline bundle:
1. pip lists EVERY artifact of a pinned version - cffi 2.1.0 alone has 100
hashes. It proves the wheel is genuine, not that it is the wheel this
bundle was built and tested with.
2. pip ignores extra files in the wheelhouse. A stale wheel left behind by
a previous build sits there unnoticed until a resolve picks it up.
3. pip says nothing about the rest of the payload - the Python installer,
the HttpPlatformHandler MSI, URL Rewrite, MySQL. Those are executables
that run as SYSTEM on the target server and were, until this file, the
only unverified thing the installer would run.
So the lock records an exact file set with a sha256 and a byte size each, and
verification is SET EQUALITY: a missing file, an unexpected extra file, or a
changed file all fail. Nothing is skipped and nothing is "close enough".
The app tree is deliberately NOT covered. It is built from the repository on
every run and changes with every commit; hashing it would make the lock churn
constantly and train everyone to regenerate it without reading it. Git is the
record for the app tree. This file is the record for everything that comes
from outside the repository.
.NOTES
Stock Windows PowerShell 5.1, and also runs under pwsh on Linux so the Bash
builder can call the same checker.
#>
# Payload directories under the bundle root. 'required' means the installer
# cannot work without it; the optional ones are per-deployment choices, and an
# absent optional directory is fine. A PRESENT directory is always checked in
# full, optional or not.
$script:BundlePayloads = @(
@{ Name = 'wheels'; Required = $true; What = 'Python wheels for the offline install' }
@{ Name = 'python'; Required = $true; What = 'the Python installer' }
@{ Name = 'httpplatformhandler'; Required = $true; What = 'the IIS module that launches waitress' }
@{ Name = 'urlrewrite'; Required = $false; What = 'IIS URL Rewrite, for the client-IP rule' }
@{ Name = 'mysql'; Required = $false; What = 'MySQL, for the bundled-database option' }
)
function Get-JsonProperty {
# shopdb-install.ps1 runs under Set-StrictMode 2.0, where reading a property
# that does not exist on a PSCustomObject THROWS instead of returning $null.
# A truncated or hand-edited bundle-lock.json would therefore blow up with
# "Property 'payloads' cannot be found" rather than saying what is wrong with
# the lock. Every read of parsed JSON goes through here.
param($Object, [string] $Name, $Default = $null)
if ($null -eq $Object) { return $Default }
$prop = $Object.PSObject.Properties[$Name]
if ($null -eq $prop) { return $Default }
return $prop.Value
}
function Get-FileDigest {
param([string] $Path)
$sha = [System.Security.Cryptography.SHA256]::Create()
$stream = [System.IO.File]::OpenRead($Path)
try { return (-join ($sha.ComputeHash($stream) | ForEach-Object { $_.ToString('x2') })) }
finally { $stream.Dispose(); $sha.Dispose() }
}
function Get-PayloadFiles {
# Every file in the directory, keyed by its path RELATIVE to that directory
# with forward slashes, so a lock generated on Windows reads the same from
# the Bash builder.
param([string] $Dir)
$out = @{}
if (-not (Test-Path $Dir)) { return $out }
$root = (Resolve-Path $Dir).Path.TrimEnd('\', '/')
foreach ($f in (Get-ChildItem $Dir -Recurse -File)) {
$rel = $f.FullName.Substring($root.Length).TrimStart('\', '/').Replace('\', '/')
$out[$rel] = @{ sha256 = (Get-FileDigest $f.FullName); size = $f.Length }
}
return $out
}
function New-BundleLock {
<#
Hash a staged bundle. The caller writes the result to bundle-lock.json;
this returns the object so a caller can diff it against the committed lock
before overwriting anything.
#>
param(
[Parameter(Mandatory = $true)] [string] $BundleRoot,
[string] $PythonTag = 'cp314',
[string] $Platform = 'win_amd64'
)
$payloads = @{}
foreach ($p in $script:BundlePayloads) {
$dir = Join-Path $BundleRoot $p.Name
if (-not (Test-Path $dir)) {
if ($p.Required) { throw ("payload directory is missing: {0} ({1})" -f $p.Name, $p.What) }
continue
}
$files = Get-PayloadFiles $dir
if ($files.Count -eq 0 -and $p.Required) {
throw ("payload directory is empty: {0} ({1})" -f $p.Name, $p.What)
}
$payloads[$p.Name] = @{ required = $p.Required; files = $files }
}
return [ordered]@{
schema = 1
generated = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
pythontag = $PythonTag
platform = $Platform
note = 'Exact third-party payload of the installer bundle. Regenerate with refresh-bundle-lock.ps1 and COMMIT the change as a reviewed dependency bump.'
payloads = $payloads
}
}
function Test-BundleLock {
<#
Compare a staged bundle against a lock. Returns an array of problem
strings - EMPTY means the bundle is exactly what the lock describes.
Returning problems rather than throwing is deliberate: an operator fixing a
wheelhouse wants the whole list at once, not one failure per rebuild.
#>
param(
[Parameter(Mandatory = $true)] [string] $BundleRoot,
[Parameter(Mandatory = $true)] $Lock
)
$problems = @()
$payloads = Get-JsonProperty $Lock 'payloads'
if ($null -eq $payloads) { return @('bundle-lock.json has no "payloads" section') }
foreach ($p in $script:BundlePayloads) {
$name = $p.Name
$dir = Join-Path $BundleRoot $name
$present = Test-Path $dir
$entry = Get-JsonProperty $payloads $name
$locked = ($null -ne $entry)
if (-not $locked) {
# Not in the lock at all. An unlocked directory that exists is a
# payload nobody reviewed, which is exactly what this is here to stop.
if ($present) { $problems += "$name/ is present but is not in bundle-lock.json - regenerate the lock" }
elseif ($p.Required) { $problems += "$name/ is required but is in neither the bundle nor the lock" }
continue
}
if (-not $present) {
if ($p.Required -or (Get-JsonProperty $entry 'required' $false)) {
$problems += "$name/ is in the lock but missing from the bundle ($($p.What))"
}
continue
}
$expected = @{}
$lockedFiles = Get-JsonProperty $entry 'files'
if ($null -ne $lockedFiles) {
foreach ($prop in $lockedFiles.PSObject.Properties) { $expected[$prop.Name] = $prop.Value }
}
$actual = Get-PayloadFiles $dir
# Sorted so the report reads the same way twice, and so it matches the
# order verify_bundle_lock.py produces.
foreach ($rel in ($expected.Keys | Sort-Object)) {
if (-not $actual.ContainsKey($rel)) { $problems += "$name/$rel is in the lock but missing from the bundle"; continue }
if ($actual[$rel].sha256 -ne $expected[$rel].sha256) {
$problems += "$name/$rel does NOT match the lock (expected sha256 $($expected[$rel].sha256.Substring(0,12))..., got $($actual[$rel].sha256.Substring(0,12))...)"
} elseif ([int64] $actual[$rel].size -ne [int64] $expected[$rel].size) {
# Cannot happen for a matching sha256, so it means the lock itself
# was hand-edited. Say so rather than passing it.
$problems += "$name/$rel size disagrees with the lock - the lock has been edited by hand"
}
}
foreach ($rel in ($actual.Keys | Sort-Object)) {
if (-not $expected.ContainsKey($rel)) { $problems += "$name/$rel is in the bundle but NOT in the lock (unexpected extra file)" }
}
}
return $problems
}
function Read-BundleLock {
param([Parameter(Mandatory = $true)] [string] $Path)
if (-not (Test-Path $Path)) { return $null }
return (Get-Content $Path -Raw | ConvertFrom-Json)
}