3606d8d696
An air-gapped site cannot be scanned from anywhere else, so when a CVE lands the only way to answer 'is that component here, and at what version' was to RDP in and go looking. The frontend was the real blind spot: nothing recorded which version of leaflet, dompurify, jspdf or html2canvas ends up inside the compiled SPA. scripts/generate_sbom.py emits CycloneDX 1.6 covering both ecosystems - every pin in requirements.txt with the sha256 the installer enforces, and every package in package-lock.json. Build-only npm packages are marked scope 'excluded' rather than dropped, so 'not here' stays distinguishable from 'not looked for'. Dependency edges are real: uv's '# via' comments give the Python graph and package-lock gives the npm one. Hand-rolled rather than cyclonedx-py plus cyclonedx-npm because both inputs are already pinned and committed - this is a format translation, not a scan - and because the build box may be a work PC with nothing but Python and Node. It is deterministic by construction: same inputs, byte-identical output, so regenerating does not churn. Staged into the application tree by both builders, so it installs onto the server with the app. shopdb-admin.ps1 verify reports it and searches it by component name, which is the question actually being asked. Packages appearing at several depths in package-lock (node_modules/vite and node_modules/vitest/node_modules/vite) are merged, and a copy reachable outside the dev tree makes the component count as shipped. Emitting both produced duplicate bom-refs, which CycloneDX forbids and scanners reject; getting the dev merge backwards would have hidden a shipped package from a CVE search. Not covered by bundle-lock.json on purpose: its provenance is git, not the third-party payload.
278 lines
13 KiB
PowerShell
278 lines
13 KiB
PowerShell
<#
|
|
.SYNOPSIS
|
|
Stage a lean per-site installer bundle on Windows, and verify its third-party
|
|
payload against bundle-lock.json.
|
|
|
|
.DESCRIPTION
|
|
The Windows equivalent of build-installer.sh, for a work PC with no Bash. It
|
|
does the whole job natively - it does NOT shell out to build-site.sh - so the
|
|
only tools needed are Python, Node and (to compile) Inno Setup.
|
|
|
|
What it does, in order:
|
|
|
|
1. Resolves the site's plugin closure from its profile.
|
|
2. Builds the SPA twice, because Vite compiles the base path in and it
|
|
therefore cannot be chosen at install time: once for /<alias> (subpath
|
|
deployment) and once for / (its own site).
|
|
3. Stages the backend tree: core, the chosen plugins only, and the runtime
|
|
files a deployable tree needs.
|
|
4. Writes plugins.iss so the wizard's plugin page matches the payload.
|
|
5. Copies the installer scripts from THIS directory.
|
|
6. Verifies wheels\, python\ and the MSIs against bundle-lock.json, and
|
|
FAILS if the bundle is not exactly what the lock describes.
|
|
|
|
Step 6 is the point of the script. A bundle that does not match its lock is
|
|
not shipped, and "the wheelhouse was missing" is caught here rather than by
|
|
an operator halfway through installing on a server with no network.
|
|
|
|
.PARAMETER Profile
|
|
Path to the site profile (see deploy\site-profile.example.json).
|
|
|
|
.PARAMETER RepoRoot
|
|
The repository. Defaults to four levels up from this script, which is correct
|
|
for a normal checkout.
|
|
|
|
.PARAMETER SkipFrontend
|
|
Reuse the SPA builds already staged in the bundle. For iterating on the
|
|
installer itself, where two Vite builds per run is most of the wall clock.
|
|
Never use it for a bundle you intend to ship.
|
|
|
|
.PARAMETER AllowUnlocked
|
|
Stage the bundle and report payload problems WITHOUT failing. For assembling
|
|
a bundle before its lock exists. A bundle built this way must not be shipped;
|
|
run refresh-bundle-lock.ps1, commit the lock, then build again without this.
|
|
|
|
.EXAMPLE
|
|
.\build-installer.ps1 -Profile ..\..\site-profile.example.json
|
|
.\build-installer.ps1 -Profile C:\sites\wjf.json -SkipFrontend
|
|
#>
|
|
|
|
[CmdletBinding()]
|
|
param(
|
|
[Parameter(Mandatory = $true)] [string] $Profile,
|
|
[string] $RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..\..')).Path,
|
|
[string] $BundleRoot = (Join-Path $PSScriptRoot 'bundle'),
|
|
[string] $SubpathAlias = 'shopdb',
|
|
[switch] $SkipFrontend,
|
|
[switch] $AllowUnlocked
|
|
)
|
|
|
|
$ErrorActionPreference = 'Stop'
|
|
. (Join-Path $PSScriptRoot 'bundle-lock.ps1')
|
|
|
|
function Say { param($m, $c = 'Gray') Write-Host $m -ForegroundColor $c }
|
|
function Step { param($m) Write-Host ''; Write-Host "==> $m" -ForegroundColor Cyan }
|
|
function Die { param($m, $fix = '') Write-Host ''; Write-Host " $m" -ForegroundColor Red
|
|
if ($fix) { Write-Host " $fix" -ForegroundColor Yellow }; exit 1 }
|
|
|
|
function Invoke-Tool {
|
|
# Runs a build tool and stops on a non-zero exit. npm writes progress to
|
|
# stderr on SUCCESS, so stderr alone must never be treated as failure.
|
|
param([string] $Exe, [string[]] $Arguments, [string] $WorkDir, [string] $What)
|
|
Push-Location $WorkDir
|
|
try {
|
|
$prev = $ErrorActionPreference
|
|
$ErrorActionPreference = 'Continue'
|
|
& $Exe @Arguments 2>&1 | ForEach-Object { Say " $_" 'DarkGray' }
|
|
$code = $LASTEXITCODE
|
|
$ErrorActionPreference = $prev
|
|
if ($code -ne 0) { Die "$What failed (exit $code)" }
|
|
} finally { Pop-Location }
|
|
}
|
|
|
|
if (-not (Test-Path $Profile)) { Die "profile not found: $Profile" }
|
|
if (-not (Test-Path $RepoRoot)) { Die "repo not found: $RepoRoot" }
|
|
$Profile = (Resolve-Path $Profile).Path
|
|
$RepoRoot = (Resolve-Path $RepoRoot).Path
|
|
$AppOut = Join-Path $BundleRoot 'app'
|
|
|
|
$python = (Get-Command python -ErrorAction SilentlyContinue)
|
|
if (-not $python) { $python = Get-Command py -ErrorAction SilentlyContinue }
|
|
if (-not $python) { Die 'Python not found on PATH' 'Install Python 3.14 and re-open the shell.' }
|
|
$npm = Get-Command npm.cmd -ErrorAction SilentlyContinue
|
|
if (-not $npm -and -not $SkipFrontend) { Die 'npm not found on PATH' 'Install Node, or pass -SkipFrontend to reuse the staged SPA.' }
|
|
|
|
Say ''
|
|
Say " repo : $RepoRoot"
|
|
Say " profile : $Profile"
|
|
Say " bundle : $BundleRoot"
|
|
|
|
# --- 1. plugin closure ------------------------------------------------------
|
|
# Same resolver the Linux builder uses, so both produce the same set from one
|
|
# profile instead of two implementations of the closure rules.
|
|
Step 'Resolving plugin closure'
|
|
$closure = (& $python.Source (Join-Path $RepoRoot 'scripts\resolve_plugin_closure.py') $Profile $RepoRoot)
|
|
if ($LASTEXITCODE -ne 0 -or -not $closure) { Die 'could not resolve the plugin closure from the profile' }
|
|
$closure = $closure.Trim()
|
|
Say " $closure" 'White'
|
|
|
|
# --- 2. frontend ------------------------------------------------------------
|
|
$frontend = Join-Path $RepoRoot 'frontend'
|
|
$subStaged = Join-Path $BundleRoot 'spa-subpath'
|
|
$rootStaged= Join-Path $BundleRoot 'spa-root'
|
|
|
|
if ($SkipFrontend) {
|
|
if (-not (Test-Path $subStaged) -or -not (Test-Path $rootStaged)) {
|
|
Die '-SkipFrontend was passed but no SPA build is staged' 'Run once without it.'
|
|
}
|
|
Say ''
|
|
Say ' Reusing the staged SPA builds (-SkipFrontend). NOT shippable if the frontend changed.' 'Yellow'
|
|
} else {
|
|
Step "Building the SPA for /$SubpathAlias/"
|
|
$env:SITE_PLUGINS = $closure
|
|
$env:VITE_BASE_PATH = "/$SubpathAlias/"
|
|
try { Invoke-Tool $npm.Source @('run','build','--silent') $frontend 'subpath frontend build' }
|
|
finally { Remove-Item Env:\VITE_BASE_PATH -ErrorAction SilentlyContinue }
|
|
Remove-Item $subStaged -Recurse -Force -ErrorAction SilentlyContinue
|
|
Copy-Item (Join-Path $frontend 'dist') $subStaged -Recurse -Force
|
|
# The alias is fixed at BUILD time; the installer reads this and refuses to
|
|
# publish under a different one rather than serving a page that cannot load
|
|
# its own assets.
|
|
Set-Content -Path (Join-Path $subStaged '.alias') -Value $SubpathAlias -Encoding ASCII
|
|
|
|
# Root build LAST, so frontend\dist is left in the state a developer expects.
|
|
Step 'Building the SPA for /'
|
|
try { Invoke-Tool $npm.Source @('run','build','--silent') $frontend 'root frontend build' }
|
|
finally { Remove-Item Env:\SITE_PLUGINS -ErrorAction SilentlyContinue }
|
|
Remove-Item $rootStaged -Recurse -Force -ErrorAction SilentlyContinue
|
|
Copy-Item (Join-Path $frontend 'dist') $rootStaged -Recurse -Force
|
|
}
|
|
|
|
# --- 3. backend tree --------------------------------------------------------
|
|
Step "Staging the application tree"
|
|
Remove-Item $AppOut -Recurse -Force -ErrorAction SilentlyContinue
|
|
New-Item -ItemType Directory -Path (Join-Path $AppOut 'plugins') -Force | Out-Null
|
|
|
|
Copy-Item (Join-Path $RepoRoot 'shopdb') $AppOut -Recurse -Force
|
|
foreach ($name in $closure.Split(',')) {
|
|
$src = Join-Path $RepoRoot ('plugins\' + $name.Trim())
|
|
if (-not (Test-Path $src)) { Die "plugin in the closure is not on disk: $name" }
|
|
Copy-Item $src (Join-Path $AppOut 'plugins') -Recurse -Force
|
|
}
|
|
|
|
# Runtime files a deployable tree needs beyond the Python packages. Without
|
|
# these the tree imports but cannot be run or migrated.
|
|
foreach ($f in @('wsgi.py', 'requirements.txt')) {
|
|
Copy-Item (Join-Path $RepoRoot $f) $AppOut -Force
|
|
}
|
|
Copy-Item (Join-Path $RepoRoot 'migrations') $AppOut -Recurse -Force
|
|
# ONLY web.config, not all of deploy\. The bundle is staged INSIDE deploy\, so
|
|
# copying the whole tree would recurse into its own output, and the rest of
|
|
# deploy\ is installer source that has no business on an application server.
|
|
# shopdb-install.ps1 reads it from exactly this path.
|
|
$cfgSrc = Join-Path $RepoRoot 'deploy\windows\web.config'
|
|
if (Test-Path $cfgSrc) {
|
|
New-Item -ItemType Directory -Path (Join-Path $AppOut 'deploy\windows') -Force | Out-Null
|
|
Copy-Item $cfgSrc (Join-Path $AppOut 'deploy\windows') -Force
|
|
}
|
|
|
|
# A CycloneDX SBOM of everything this tree depends on, Python and npm together.
|
|
# Staged INTO the tree so it installs onto the server with the application: an
|
|
# air-gapped site cannot be scanned remotely, so the only way to answer "are we
|
|
# exposed to this CVE, and where" is for the answer to be sitting on the box.
|
|
Step 'Generating SBOM'
|
|
& $python.Source (Join-Path $RepoRoot 'scripts\generate_sbom.py') $RepoRoot `
|
|
-o (Join-Path $AppOut 'sbom.cdx.json') | ForEach-Object { Say " $_" 'White' }
|
|
if ($LASTEXITCODE -ne 0) { Die 'SBOM generation failed' }
|
|
|
|
# Stage the profile INTO the tree: `flask plugin apply-profile` at provisioning
|
|
# reads the same profile the tree was staged from, so the installed plugin set
|
|
# and the shipped plugin code cannot drift.
|
|
Copy-Item $Profile (Join-Path $AppOut 'site-profile.json') -Force
|
|
|
|
# The installer expects frontend\dist and frontend\dist-subpath.
|
|
$feOut = Join-Path $AppOut 'frontend'
|
|
New-Item -ItemType Directory -Path $feOut -Force | Out-Null
|
|
Copy-Item $rootStaged (Join-Path $feOut 'dist') -Recurse -Force
|
|
Copy-Item $subStaged (Join-Path $feOut 'dist-subpath') -Recurse -Force
|
|
|
|
Get-ChildItem $AppOut -Recurse -Directory -Filter '__pycache__' -ErrorAction SilentlyContinue |
|
|
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue
|
|
Get-ChildItem $AppOut -Recurse -File -Filter '*.pyc' -ErrorAction SilentlyContinue |
|
|
Remove-Item -Force -ErrorAction SilentlyContinue
|
|
|
|
# --- 4. plugins.iss ---------------------------------------------------------
|
|
Step 'Writing plugins.iss'
|
|
$shipped = (Get-ChildItem (Join-Path $AppOut 'plugins') -Directory | Select-Object -ExpandProperty Name) -join ','
|
|
$aliasBuilt = ''
|
|
$aliasFile = Join-Path $feOut 'dist-subpath\.alias'
|
|
if (Test-Path $aliasFile) { $aliasBuilt = (Get-Content $aliasFile -TotalCount 1).Trim() }
|
|
@"
|
|
; GENERATED by build-installer.ps1 - do not edit.
|
|
; The plugins present in bundle\app\plugins at build time.
|
|
#define AvailablePlugins "$shipped"
|
|
; The alias the subpath SPA was built for, or empty if this bundle has no
|
|
; subpath build - in which case the wizard must not offer that option.
|
|
#define SubpathAlias "$aliasBuilt"
|
|
"@ | Set-Content -Path (Join-Path $PSScriptRoot 'plugins.iss') -Encoding ASCII
|
|
Say " $shipped" 'White'
|
|
|
|
# --- 5. installer scripts ---------------------------------------------------
|
|
# From THIS directory, which is the reviewed copy under version control. They
|
|
# used to be copied from a downloads folder, so the logic that shipped was not
|
|
# the logic that was committed and the build only worked on one machine.
|
|
Step 'Copying installer scripts'
|
|
foreach ($f in @('shopdb-install.ps1', 'shopdb-preflight.ps1', 'bundle-lock.ps1')) {
|
|
$src = Join-Path $PSScriptRoot $f
|
|
if (-not (Test-Path $src)) { Die "installer script missing from the repo: $f" }
|
|
Copy-Item $src $BundleRoot -Force
|
|
Say " $f"
|
|
}
|
|
|
|
# --- 6. payload verification ------------------------------------------------
|
|
Step 'Verifying the third-party payload against bundle-lock.json'
|
|
$lockPath = Join-Path $PSScriptRoot 'bundle-lock.json'
|
|
$lock = Read-BundleLock $lockPath
|
|
if (-not $lock) {
|
|
$msg = "no bundle-lock.json at $lockPath"
|
|
if ($AllowUnlocked) { Say " $msg - continuing because -AllowUnlocked was passed" 'Yellow' }
|
|
else {
|
|
Die $msg @'
|
|
Add the wheels and installers to the bundle, then:
|
|
.\refresh-bundle-lock.ps1 (review the list)
|
|
.\refresh-bundle-lock.ps1 -Yes (write it)
|
|
and commit bundle-lock.json. Pass -AllowUnlocked to stage a bundle without one -
|
|
it must not be shipped.
|
|
'@
|
|
}
|
|
} else {
|
|
$problems = Test-BundleLock -BundleRoot $BundleRoot -Lock $lock
|
|
if ($problems.Count -eq 0) {
|
|
Say (" payload matches the lock ({0}, {1})" -f `
|
|
(Get-JsonProperty $lock 'pythontag' 'unknown'), (Get-JsonProperty $lock 'platform' 'unknown')) 'Green'
|
|
# Ships WITH the bundle: the installer re-checks the payload on the
|
|
# target server before running any of it, so tampering between build and
|
|
# install is caught too.
|
|
Copy-Item $lockPath $BundleRoot -Force
|
|
} else {
|
|
Say ''
|
|
foreach ($p in $problems) { Say " $p" 'Red' }
|
|
Say ''
|
|
if ($AllowUnlocked) {
|
|
Say ' -AllowUnlocked: continuing anyway. DO NOT SHIP this bundle.' 'Yellow'
|
|
} else {
|
|
Die ("{0} payload problem(s) - the bundle is not what the lock describes" -f $problems.Count) @'
|
|
Either the payload is wrong (fix the bundle) or it changed on purpose (run
|
|
refresh-bundle-lock.ps1, read the diff, and commit the new lock).
|
|
'@
|
|
}
|
|
}
|
|
}
|
|
|
|
# --- summary ----------------------------------------------------------------
|
|
Write-Host ''
|
|
Say " Bundle staged at: $BundleRoot" 'White'
|
|
foreach ($d in @('app', 'wheels', 'python', 'httpplatformhandler', 'urlrewrite', 'mysql')) {
|
|
$p = Join-Path $BundleRoot $d
|
|
if (Test-Path $p) {
|
|
$mb = ((Get-ChildItem $p -Recurse -File | Measure-Object Length -Sum).Sum / 1MB)
|
|
Say (" {0,-22} {1,8:N1} MB" -f $d, $mb)
|
|
} else {
|
|
Say (" {0,-22} {1}" -f $d, 'absent') 'DarkGray'
|
|
}
|
|
}
|
|
Write-Host ''
|
|
Say ' Compile with: iscc ShopDBFlask.iss' 'White'
|
|
Say ' (Inno Setup 6.6.0 or newer - the wizard uses the windows11 custom style.)' 'DarkGray'
|
|
Write-Host ''
|