feat(installer): bundle URL Rewrite, ask where client IPs come from, verify installs

IIS does not set X-Forwarded-For on its own and HttpPlatformHandler connects
from loopback, so without a rewrite rule every client reads as 127.0.0.1. The
GE-Enforce IP allowlist, the dashboard visitor-location lookup and per-host
login rate limiting all stop working, silently. The rule needed URL Rewrite,
which the installer told operators to download - from an air-gapped server.

URL Rewrite now ships in the bundle, and the wizard asks which case applies,
because the two answers are mutually exclusive. Directly exposed: install it and
set X-Forwarded-For from REMOTE_ADDR, which is what stops a client spoofing its
own. Behind a proxy: leave the rule off, since REMOTE_ADDR is the proxy and
applying it would discard the real client IP.

The rule is enabled by deleting two explicit marker lines rather than by a regex
over the surrounding comment, so editing that prose cannot silently disable it.

An existing web.config is no longer overwritten. It is the one file on a server
that legitimately carries hand-edits, and replacing it reverted them without a
word - on a server where the X-Forwarded-For rule had been enabled by hand, that
alone would have turned the GE-Enforce IP allowlist off. The installer reports
what it found instead.

pip now runs with --require-hashes and --only-binary=:all:. Hash-checking is
requested explicitly rather than inferred from the lockfile, so shipping an
unhashed requirements.txt fails loudly instead of quietly dropping the check.

shopdb-admin.ps1 gains a verify command: which bundle this server was installed
from, and whether the installed packages still match what shipped.

The .iss states its compiler floor. WizardStyle uses the built-in windows11
custom style, which needs Inno Setup 6.6.0; older compilers now fail with that
sentence rather than 'WizardStyle is invalid'.
This commit is contained in:
cproudlock
2026-08-03 11:17:58 -04:00
parent 88af7fd9ce
commit 44237b5cbd
5 changed files with 286 additions and 19 deletions

View File

@@ -9,6 +9,14 @@
; that nobody tests. This file only collects operator input, runs the stages in
; order, and reports which one failed.
; Inno Setup 6.6.0 is the floor: WizardStyle below uses the built-in 'windows11'
; custom style, which 6.5 and earlier reject. Stated here so a build box on an
; older compiler fails with this sentence rather than with a bare
; "Value of [Setup] section directive WizardStyle is invalid".
#if VER < EncodeVer(6,6,0)
#error This script needs Inno Setup 6.6.0 or newer (WizardStyle=... windows11). Download it from https://jrsoftware.org/isdl.php
#endif
#define AppName "ShopDB-Flask"
; Pre-release. This has not shipped, so it is 0.x by definition. It becomes
; 1.0.0 when a real site installs from it successfully - not before.
@@ -173,6 +181,7 @@ var
DbPageReady: Boolean;
DeployPage: TInputOptionWizardPage;
DeployPageReady: Boolean;
ClientIpPage: TInputOptionWizardPage;
// Set from the streamed stage output so a failure can name its cause. Without
// this the wizard could only report "exit 1", which points at nothing.
FailCause: String;
@@ -388,6 +397,35 @@ begin
else
SitePage.Values[0] := GetComputerNameString;
SitePage.Values[1] := '8090';
// Where the real client IP comes from. Not cosmetic: IIS sends no
// X-Forwarded-For of its own, so with neither option applied every request
// reads as 127.0.0.1 and the GE-Enforce IP allowlist, the dashboard's
// visitor-location lookup and per-host login rate limiting all fail silently.
//
// The two answers are mutually exclusive, and picking the wrong one is worse
// than picking neither: the rule OVERWRITES the header with REMOTE_ADDR, which
// is exactly right when IIS faces clients (it defeats spoofing) and exactly
// wrong behind a proxy (REMOTE_ADDR is the proxy, so the real client IP is
// discarded). Hence a question rather than a default.
ClientIpPage := CreateInputOptionPage(SitePage.ID,
'Client addresses', 'How does this server see who is connecting?',
'ShopDB records the address of every request, and some features decide what '
+ 'to show based on it. Choose whichever describes this server.',
True, False);
ClientIpPage.Add('Clients connect to this server directly (installs URL Rewrite)');
ClientIpPage.Add('A proxy or load balancer sits in front of this server');
ClientIpPage.SelectedValueIndex := 0;
end;
// 'direct' installs URL Rewrite from the bundle and enables the X-Forwarded-For
// rule; 'proxy' leaves both alone because the proxy already sets the header.
function ClientIpSourceArg: String;
begin
if ClientIpPage.SelectedValueIndex = 1 then
Result := 'proxy'
else
Result := 'direct';
end;
// Full path to the 64-bit PowerShell.
@@ -900,6 +938,7 @@ begin
' -SiteHost "' + SitePage.Values[0] + '"' +
' -SitePort ' + SitePage.Values[1] +
' -OnFailure never' +
' -ClientIpSource ' + ClientIpSourceArg +
' -SitePlugins "' + SelectedPlugins + '"';
// Subpath deployment: an IIS Application under the existing site instead of a

View File

@@ -22,7 +22,7 @@
[CmdletBinding()]
param(
[ValidateSet('menu','status','start','stop','restart','logs','open','backup',
'check','sessions','plugins','add-plugin','uninstall')]
'check','sessions','plugins','add-plugin','verify','uninstall')]
[string] $Command = 'menu',
[string] $Path = '',
[string] $AppRoot = 'C:\shopdb-flask',
@@ -473,6 +473,69 @@ function Add-Plugin {
Say (" {0} added" -f $Name) 'Green'
}
function Invoke-Verify {
<#
Prove the installed dependencies are still the ones that shipped.
Two independent records, both written at install time:
requirements.txt a sha256 for every wheel, checked by pip on install
bundle-lock.json the exact third-party payload of the bundle
This re-checks what can still be checked on a live server. It reports; it
changes nothing.
#>
Head 'Verify'
$lock = Join-Path $AppRoot 'bundle-lock.json'
if (Test-Path $lock) {
try {
$l = Get-Content $lock -Raw | ConvertFrom-Json
Say (" installed from : {0} bundle ({1}, {2})" -f $l.generated, $l.pythontag, $l.platform)
} catch { Say ' bundle-lock.json is present but unreadable' 'Yellow' }
} else {
Say ' no bundle-lock.json recorded (installed before payload locking, or by hand)' 'Yellow'
}
# pip's own audit. It re-reads the metadata of what is actually installed and
# reports anything missing or version-inconsistent, which is the part that
# can still drift after install - a hand-run `pip install` on the server.
$py = Join-Path $AppRoot 'venv\Scripts\python.exe'
if (-not (Test-Path $py)) { Say ' no venv - application not installed' 'Red'; return }
Say ''
Say ' checking installed packages against requirements.txt...'
$req = Join-Path $AppRoot 'requirements.txt'
if (-not (Test-Path $req)) { Say ' requirements.txt is missing from the install' 'Red'; return }
$wanted = @{}
foreach ($line in (Get-Content $req)) {
if ($line -match '^([A-Za-z0-9._-]+)==([^ \\]+)') { $wanted[$Matches[1].ToLower().Replace('_','-')] = $Matches[2] }
}
$frozen = @{}
foreach ($line in (& $py -m pip freeze 2>$null)) {
if ($line -match '^([A-Za-z0-9._-]+)==(.+)$') { $frozen[$Matches[1].ToLower().Replace('_','-')] = $Matches[2] }
}
$bad = 0
foreach ($name in ($wanted.Keys | Sort-Object)) {
if (-not $frozen.ContainsKey($name)) {
Say (" MISSING {0} {1}" -f $name, $wanted[$name]) 'Red'; $bad++
} elseif ($frozen[$name] -ne $wanted[$name]) {
Say (" DIFFERS {0} {1} installed, {2} expected" -f $name, $frozen[$name], $wanted[$name]) 'Red'; $bad++
}
}
if ($bad -eq 0) {
Say (" all {0} packages match" -f $wanted.Count) 'Green'
} else {
Say ''
Say (" {0} package(s) do not match what shipped." -f $bad) 'Red'
Say ' Something was installed or upgraded on this server by hand. Re-run the' 'Yellow'
Say ' installer to put the shipped set back.' 'Yellow'
}
& $py -m pip check 2>&1 | ForEach-Object { Say " $_" 'DarkGray' }
}
function Show-Uninstall {
Head 'Uninstall'
Say ' Use Settings > Apps > ShopDB-Flask, or Add/Remove Programs.'
@@ -492,13 +555,15 @@ function Show-Menu {
Write-Host ' 2 Stop the application 7 Worker processes' -ForegroundColor White
Write-Host ' 3 Start the application 8 Open in browser' -ForegroundColor White
Write-Host ' 4 View recent logs 9 Plugins' -ForegroundColor White
Write-Host ' 5 Health check 0 Exit' -ForegroundColor White
Write-Host ' 5 Health check V Verify this install' -ForegroundColor White
Write-Host ' 0 Exit' -ForegroundColor White
Write-Host ''
$c = Read-Host ' Choose'
switch ($c) {
'1' { Restart-App } '2' { Stop-App } '3' { Start-App }
'4' { Show-Logs } '5' { Invoke-Check } '6' { Backup-Db $Path }
'7' { Show-Sessions } '8' { Open-Site }
'v' { Invoke-Verify } 'V' { Invoke-Verify }
'9' { Show-Plugins
$add = Read-Host ' Name of a shipped plugin to add (Enter to skip)'
if ($add) { Add-Plugin $add } }
@@ -524,6 +589,7 @@ switch ($Command) {
'sessions' { Show-Sessions }
'plugins' { Show-Plugins }
'add-plugin'{ Add-Plugin $Path }
'verify' { Invoke-Verify }
'uninstall' { Show-Uninstall }
default { Show-Menu }
}

View File

@@ -23,7 +23,13 @@
wheels\*.whl
app\ (release tree: wsgi.py, shopdb\, plugins\, frontend\dist\, ...)
httpplatformhandler\httpPlatformHandler_amd64.msi
urlrewrite\ (optional, -ClientIpSource direct)
mysql\ (optional, bundled-database option)
bundle-lock.json + bundle-lock.ps1
The payload is checked against bundle-lock.json before anything runs. A
bundle whose wheels or MSIs do not match the lock exactly - missing, extra or
altered - is refused, because every one of them executes as SYSTEM here.
.PARAMETER DbHost
Existing-MySQL option: the server hostname. The PASSWORD IS NEVER A PARAMETER -
@@ -102,6 +108,19 @@ param(
# installer checks that and refuses if they disagree.
[string] $MountAlias = '',
[string] $ParentSite = 'Default Web Site',
# How this server learns a request's real client IP.
#
# direct IIS is exposed to clients. Install URL Rewrite from the bundle
# and set X-Forwarded-For from REMOTE_ADDR. Overwriting the header
# is what stops a client spoofing it.
# proxy A reverse proxy (ARR, load balancer) sits in front and already
# sets X-Forwarded-For. Leave the rule off - REMOTE_ADDR would be
# the proxy, so applying it would DESTROY the real client IP.
#
# Without one or the other, IIS sends no X-Forwarded-For at all and every
# client reads as 127.0.0.1: the GE-Enforce IP allowlist, the dashboard
# visitor-location lookup and per-host login rate limiting all break quietly.
[ValidateSet('direct','proxy')] [string] $ClientIpSource = 'direct',
[switch] $WhatIfOnly,
# Unattended runs cannot answer a prompt. Choose the failure behaviour up front.
[ValidateSet('ask','always','never')] [string] $OnFailure = 'ask'
@@ -236,6 +255,45 @@ $AppSource = Join-Path $BundleRoot 'app'
# - the database is backed up BEFORE migrations touch it;
# - a failed migration puts the backup back rather than leaving a half-state.
function Assert-BundleIntegrity {
<#
Refuse to run against a payload that is not exactly what was built and
reviewed.
This runs BEFORE anything is installed, because everything downstream is an
executable that runs as SYSTEM on this server: the Python installer, the
HttpPlatformHandler and URL Rewrite MSIs, and ~40 wheels. requirements.txt
hashes cover the wheels once pip gets to them; nothing covered the rest,
and nothing noticed a stale extra wheel sitting in the wheelhouse.
There is deliberately no override. A bundle that fails here was altered
after it was built, and the fix is to get a correct bundle rather than to
wave this one through on a server nobody can see.
#>
$checker = Join-Path $BundleRoot 'bundle-lock.ps1'
$lockFile = Join-Path $BundleRoot 'bundle-lock.json'
if (-not (Test-Path $checker)) {
Fail 'bundle-lock.ps1 is missing from the bundle' `
'This bundle was not assembled by build-installer.ps1/.sh. Rebuild it.'
}
if (-not (Test-Path $lockFile)) {
Fail 'bundle-lock.json is missing from the bundle' `
'This bundle was not assembled by build-installer.ps1/.sh. Rebuild it.'
}
. $checker
$lock = Read-BundleLock $lockFile
$problems = Test-BundleLock -BundleRoot $BundleRoot -Lock $lock
if ($problems.Count -gt 0) {
foreach ($p in $problems) { Write-Log " $p" 'FAIL' }
Fail ("the bundle does not match bundle-lock.json ({0} problem(s))" -f $problems.Count) @'
The payload was changed after this installer was built. Do not install from it.
Obtain a bundle whose payload matches its lock, or rebuild one and re-compile.
'@
}
Write-Log ("bundle payload verified against bundle-lock.json ({0}, {1})" -f `
(Get-JsonProperty $lock 'pythontag' 'unknown'), (Get-JsonProperty $lock 'platform' 'unknown')) 'OK'
}
function Get-BundleVersion {
# The version being installed, read from the payload itself so it can never
# disagree with the code that is about to be copied.
@@ -642,8 +700,9 @@ function Invoke-Stage2 {
Write-Log 'STAGE 2: runtime and configuration' 'STEP'
if (-not (Test-Path $BundleRoot)) { Fail "bundle not found: $BundleRoot" }
if (-not (Test-Path $WheelDir)) { Fail "wheelhouse not found: $WheelDir" 'Build it on Windows with the matching Python. See build-offline-bundle-native.ps1.' }
if (-not (Test-Path $WheelDir)) { Fail "wheelhouse not found: $WheelDir" 'Rebuild the bundle with deploy\windows\installer\build-installer.ps1 (or .sh); it refuses to produce a bundle without one.' }
if (-not (Test-Path $AppSource)) { Fail "application payload not found: $AppSource" }
Assert-BundleIntegrity
# --- Python, all users --------------------------------------------------
# A per-user install lands in %LOCALAPPDATA%, which the IIS app-pool identity
@@ -765,6 +824,12 @@ a page that cannot load its own assets. Rebuild with scripts/build-site.sh
Set-Content -Path (Join-Path $AppRoot '.installed-version') `
-Value $script:BundleVersion -Encoding ASCII
}
# Keep the lock with the install so `shopdb-admin.ps1 verify` can say WHICH
# bundle this server was built from, months later and offline.
if (-not $WhatIfOnly) {
$bundledLock = Join-Path $BundleRoot 'bundle-lock.json'
if (Test-Path $bundledLock) { Copy-Item $bundledLock $AppRoot -Force }
}
foreach ($sub in @('logs','instance')) {
$p = Join-Path $AppRoot $sub
if (-not (Test-Path $p)) { New-Item -ItemType Directory -Path $p -Force | Out-Null }
@@ -784,13 +849,27 @@ a page that cannot load its own assets. Rebuild with scripts/build-site.sh
# --- offline dependency install ----------------------------------------
# PIP_NO_INDEX makes a network attempt impossible rather than merely
# unnecessary. On an air-gapped box pip otherwise hangs on DNS timeouts.
Write-Log 'installing dependencies from the wheelhouse (offline)'
#
# --require-hashes puts pip in hash-checking mode: every wheel must match a
# sha256 listed in requirements.txt or the install ABORTS. Without it pip
# took whatever file in the wheelhouse satisfied the version pin, so a
# swapped or hand-dropped wheel installed silently. The flag is passed
# explicitly rather than relying on pip inferring it from the presence of
# hashes, so shipping an unhashed requirements.txt fails loudly here instead
# of quietly dropping the check.
#
# --only-binary=:all: refuses to fall back to building from an sdist. On a
# server with no compiler and no network that fallback cannot succeed; it
# just turns a clear "no wheel for this platform" into a confusing build
# error deep in someone's setup.py.
Write-Log 'installing dependencies from the wheelhouse (offline, hash-checked)'
if (-not $WhatIfOnly) {
$env:PIP_NO_INDEX = '1'
$env:PIP_FIND_LINKS = $WheelDir
$env:PIP_DISABLE_PIP_VERSION_CHECK = '1'
try {
Invoke-Native $Py @('-m','pip','install','--no-index',"--find-links=$WheelDir",
'--require-hashes','--only-binary=:all:',
'-r',(Join-Path $AppRoot 'requirements.txt')) 'dependency install'
} finally {
Remove-Item Env:\PIP_NO_INDEX, Env:\PIP_FIND_LINKS -ErrorAction SilentlyContinue
@@ -1230,18 +1309,83 @@ the module by hand, then re-run stage 4.
}
}
# web.config ships in the repo; only its paths need correcting.
# --- client IP -----------------------------------------------------------
# URL Rewrite is what lets the XFF rule below exist. Install it BEFORE
# writing a web.config that references <rewrite>, or IIS answers every
# request with 500.19 until someone works out which module is missing.
if ($ClientIpSource -eq 'direct') {
$rewriteDll = Join-Path $env:windir 'system32\inetsrv\rewrite.dll'
if (Test-Path $rewriteDll) {
Write-Log 'URL Rewrite already installed' 'OK'
} else {
$rwMsi = Get-ChildItem (Join-Path $BundleRoot 'urlrewrite') -Filter '*.msi' -ErrorAction SilentlyContinue |
Select-Object -First 1
if (-not $rwMsi) {
Fail 'URL Rewrite is not installed and no MSI is in the bundle' @'
-ClientIpSource direct needs the IIS URL Rewrite module to set X-Forwarded-For.
Add urlrewrite\rewrite_amd64.msi to the bundle and rebuild, or re-run with
-ClientIpSource proxy if something in front of IIS already sets the header.
Installing it later by hand also works: this server has no network, so the MSI
has to come from the bundle either way.
'@
}
Write-Log "installing $($rwMsi.Name)"
if (-not $WhatIfOnly) {
Invoke-Native 'msiexec.exe' @('/i', $rwMsi.FullName, '/quiet', '/norestart') `
'URL Rewrite MSI' -TimeoutSec 600
if (-not (Test-Path $rewriteDll)) {
Fail 'the URL Rewrite MSI reported success but the module is missing' `
'An installer exit code of 0 does not prove it did what was asked.'
}
Write-Log 'URL Rewrite installed' 'OK'
}
}
} else {
Write-Log 'client IP comes from an upstream proxy; not installing URL Rewrite' 'OK'
}
# web.config ships in the repo; its paths need correcting and the client-IP
# rule needs enabling or leaving off.
#
# An EXISTING web.config is never overwritten. It is the one file on the
# server that legitimately carries hand-edits - a nested application for
# /installers, bindings, a proxy-specific rule - and overwriting it reverts
# them silently. On a server where the XFF rule was enabled by hand, that
# alone would turn the GE-Enforce IP allowlist off without a word in any log.
$srcCfg = Join-Path $AppRoot 'deploy\windows\web.config'
$dstCfg = Join-Path $AppRoot 'web.config'
if (Test-Path $srcCfg) {
Write-Log 'installing web.config'
if (Test-Path $dstCfg) {
Write-Log 'web.config already exists; leaving it alone' 'OK'
$existing = Get-Content $dstCfg -Raw
$hasRule = $existing -match '<rewrite>' -and $existing -notmatch 'SHOPDB-CLIENTIP-BEGIN'
if ($ClientIpSource -eq 'direct' -and -not $hasRule) {
Write-Log 'this web.config does NOT set X-Forwarded-For; client IPs will read as 127.0.0.1' 'WARN'
Write-Log ' enable the SHOPDB-CLIENTIP block by hand, or delete web.config and re-run stage 4' 'WARN'
}
if ($ClientIpSource -eq 'proxy' -and $hasRule) {
Write-Log 'this web.config OVERWRITES X-Forwarded-For, which discards the real client IP behind a proxy' 'WARN'
Write-Log ' remove the <rewrite> block by hand if a proxy in front already sets the header' 'WARN'
}
} elseif (Test-Path $srcCfg) {
Write-Log "installing web.config (client IP: $ClientIpSource)"
if (-not $WhatIfOnly) {
$cfg = Get-Content $srcCfg -Raw
$cfg = $cfg.Replace('C:\shopdb-flask', $AppRoot)
if ($ClientIpSource -eq 'direct') {
# Uncomment by deleting the two marker lines. A string operation
# on explicit markers, not a regex over the surrounding prose:
# editing that prose must never silently disable the rule.
if ($cfg -notmatch 'SHOPDB-CLIENTIP-BEGIN') {
Fail 'the shipped web.config has no SHOPDB-CLIENTIP block' `
'It was edited. Restore deploy\windows\web.config from the repository.'
}
$cfg = $cfg.Replace('<!-- SHOPDB-CLIENTIP-BEGIN', '').Replace('SHOPDB-CLIENTIP-END -->', '')
Write-Log 'X-Forwarded-For rule enabled' 'OK'
}
Set-Content -Path $dstCfg -Value $cfg -Encoding UTF8
Track 'file' $dstCfg
}
} elseif (-not (Test-Path $dstCfg)) {
} else {
Fail "web.config not found at $srcCfg or $dstCfg"
}

View File

@@ -306,10 +306,15 @@ Invoke-Check 'IIS' 'URL Rewrite module' {
}
$modules = & $appcmd list module 2>$null
if ($modules -match 'RewriteModule') {
Add-Result 'IIS' 'URL Rewrite module' 'PASS' 'installed (X-Forwarded-For rule can be enabled)'
Add-Result 'IIS' 'URL Rewrite module' 'PASS' 'installed (the X-Forwarded-For rule can be enabled)'
} else {
# Not a FAIL: it is only needed for -ClientIpSource direct, and the
# installer carries the MSI and installs it itself. Worth reporting
# because without the rule IIS sends no X-Forwarded-For at all, so every
# client reads as 127.0.0.1 and the GE-Enforce IP allowlist, the
# visitor-location lookup and per-host login rate limiting go quiet.
Add-Result 'IIS' 'URL Rewrite module' 'INFO' 'not installed' `
'Optional. Leave the <rewrite> block in web.config commented out, or IIS returns 500.19.'
'The installer installs it from the bundle when -ClientIpSource is direct. Behind a reverse proxy that already sets X-Forwarded-For, use -ClientIpSource proxy and leave it out.'
}
}

View File

@@ -49,16 +49,29 @@
</httpPlatform>
<!--
OPTIONAL: forward the real client IP so audit logs and the kiosk
visitor-location feature (IP -> business unit) see the caller, not the
loopback that HttpPlatformHandler connects from.
Forward the real client IP, so the audit log, the kiosk visitor-location
feature (IP -> business unit), the GE-Enforce IP allowlist and per-host
login rate limiting all see the caller rather than the loopback address
HttpPlatformHandler connects from.
This block is COMMENTED OUT by default because it needs the URL Rewrite
module; with it uncommented but URL Rewrite not installed, IIS returns
HTTP 500.19 ("configuration section not well-formed / cannot be read").
Install URL Rewrite (https://www.iis.net/downloads/microsoft/url-rewrite)
and then uncomment the <rewrite> block below to enable it.
IIS does not set X-Forwarded-For on its own. Without the rule below there
is no such header at all, and every client looks like 127.0.0.1 - so the
allowlist and the visitor-location lookup silently stop working.
ONLY CORRECT WHEN IIS IS DIRECTLY EXPOSED. It overwrites the header with
REMOTE_ADDR, which is what stops a client spoofing its own X-Forwarded-For.
Behind a reverse proxy (ARR, a load balancer) REMOTE_ADDR is the PROXY, so
this rule would destroy the real client IP - there, leave it disabled and
let the proxy set the header.
It ships DISABLED because it needs the URL Rewrite module; enabled without
it, IIS returns HTTP 500.19 ("configuration section not well-formed").
The installer handles both: -ClientIpSource direct installs URL Rewrite
from the bundle and enables this; -ClientIpSource proxy leaves it alone.
By hand: install URL Rewrite, then delete the two marker lines below.
-->
<!-- SHOPDB-CLIENTIP-BEGIN
<rewrite>
<allowedServerVariables>
<add name="HTTP_X_FORWARDED_FOR" />
@@ -73,7 +86,7 @@
</rule>
</rules>
</rewrite>
-->
SHOPDB-CLIENTIP-END -->
</system.webServer>