diff --git a/CLAUDE.md b/CLAUDE.md index 846c7fd..4a4bc3b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -50,6 +50,7 @@ Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely - 13 bundled plugins all satisfy contract: computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty - Core Alembic chain: baseline `68b3947ae14f` -> head `7d26_settings_description_text` (33 core migrations). Each plugin owns its own chain (ADR-008); deploy runs `flask db upgrade` then `flask plugin upgrade-all`. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8). - Lean per-site builds (ADR-013 + ADR-014): `scripts/build-site.sh` (backend) + `SITE_PLUGINS` via `scripts/stage-frontend.mjs` (frontend) ship only chosen plugins; `flask plugin prune-schema` drops non-installed plugins' tables at provisioning. Sidebar nav / settings / Displays all gate on staged routes. Manifest-less `plugins//frontend/` dirs (e.g. `applications`) are core and always ship. +- Windows sites install from a single air-gapped installer `.exe` built per site from its plugin profile (`deploy/windows/installer/`, built by `build-installer.sh` or `build-installer.ps1`). Operator docs: `docs/INSTALL-WINDOWS.md` + `docs/OPERATE-WINDOWS.md` - these are canonical for a NEW site. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are the MANUAL procedure, kept for hand-built servers only. The installer verifies its third-party payload against `bundle-lock.json` and installs wheels with `pip --require-hashes`; every build stages a CycloneDX SBOM (`sbom.cdx.json`) onto the server. - Legacy import: `docs/IMPORT-API.md` is the schema-agnostic import contract; `docs/IMPORT-ADOPTION.md` + `docs/PILOT-DEPLOY.md` cover adopting a site; `scripts/site_imports/wjf/` is the West Jefferson reference loader (all 16 stages, validated end-to-end including on a Windows + MySQL 8 VM). - API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps). - Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0 diff --git a/README.md b/README.md index 378ac33..b0535ba 100644 --- a/README.md +++ b/README.md @@ -174,8 +174,15 @@ token plus [docs/IMPORT-API.md](docs/IMPORT-API.md) drive the whole migration through documented endpoints (`X-Import-Mode` preserves original timestamps). `scripts/site_imports/wjf/` is the West Jefferson reference loader. -For the full per-site deployment runbook see [docs/DEPLOY.md](docs/DEPLOY.md); -for every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md). +### Which deployment route + +| Target | Use | +|---|---| +| **Windows Server + IIS** (how sister sites run) | **[docs/INSTALL-WINDOWS.md](docs/INSTALL-WINDOWS.md)** - one installer `.exe`, offline, no manual IIS work. Day 2: [docs/OPERATE-WINDOWS.md](docs/OPERATE-WINDOWS.md) | +| Linux / Docker, air-gapped | [docs/DEPLOY-AIRGAP.md](docs/DEPLOY-AIRGAP.md) | +| Linux / Docker, connected | [docs/DEPLOY.md](docs/DEPLOY.md) | + +For every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md). ## Configuration diff --git a/deploy/windows/installer/ShopDBFlask.iss b/deploy/windows/installer/ShopDBFlask.iss index c23daab..17c81f7 100644 --- a/deploy/windows/installer/ShopDBFlask.iss +++ b/deploy/windows/installer/ShopDBFlask.iss @@ -93,7 +93,10 @@ Name: "english"; MessagesFile: "compiler:Default.isl" WelcomeLabel1=Set up ShopDB-Flask on this server ; Short, and says the two things an operator actually wants up front: it will not ; need the internet, and it will tell them before it changes anything. -WelcomeLabel2=Everything needed is included in this installer. No internet connection is used at any point.%n%nYou will be asked four short questions, and nothing on this server is changed until you reach the final confirmation.%n%nThe next page checks that this server is ready. It only reads - it changes nothing. +; The installer supplies Python, the wheels, the IIS modules and optionally MySQL. +; It does NOT install the IIS Web Server role itself - that is a Windows feature, +; and the check on the next page confirms it is present rather than adding it. +WelcomeLabel2=Everything this application needs is included - Python, its packages, the IIS modules and optionally MySQL. No internet connection is used at any point.%n%nThis server must already have the IIS Web Server role installed. The next page checks that, and everything else this needs, before anything is changed.%n%nYou will then be asked a few short questions. Nothing on this server is changed until the final confirmation. ClickNext=Click Next to check this server. FinishedHeadingLabel=ShopDB-Flask is ready SetupAppTitle=ShopDB-Flask Setup @@ -131,18 +134,28 @@ Source: "swatch-bad.bmp"; Flags: dontcopy [Icons] ; A folder rather than loose icons: this is a server tool, not a desktop app. +; +; EVERY shortcut passes -AppRoot and -SitePort. The console defaults to +; C:\shopdb-flask and port 8090, so on any install that chose a different +; directory or port it looked in the wrong place and reported a perfectly healthy +; site as broken - from a Start Menu shortcut the installer wrote itself. Name: "{group}\ShopDB-Flask Console"; Filename: "powershell.exe"; \ - Parameters: "-NoExit -NoProfile -ExecutionPolicy Bypass -File ""{app}\shopdb-admin.ps1"""; \ + Parameters: "-NoExit -NoProfile -ExecutionPolicy Bypass -File ""{app}\shopdb-admin.ps1"" -AppRoot ""{app}"" -SitePort {code:SitePortValue}"; \ WorkingDir: "{app}"; IconFilename: "{app}\shopdb.ico"; \ Comment: "Status, restart, logs and backups" -Name: "{group}\Open ShopDB-Flask"; Filename: "http://localhost:8090/login"; \ - IconFilename: "{app}\shopdb.ico"; Comment: "Open the application in a browser" +; Resolves the address from .env at click time rather than baking one in. The +; literal http://localhost:8090/login this used to carry was wrong for every +; subpath install and every non-default port, with no operator mistake involved. +Name: "{group}\Open ShopDB-Flask"; Filename: "powershell.exe"; \ + Parameters: "-NoProfile -ExecutionPolicy Bypass -WindowStyle Hidden -File ""{app}\shopdb-admin.ps1"" open -AppRoot ""{app}"" -SitePort {code:SitePortValue}"; \ + WorkingDir: "{app}"; IconFilename: "{app}\shopdb.ico"; \ + Comment: "Open the application in a browser" Name: "{group}\Restart ShopDB-Flask"; Filename: "powershell.exe"; \ - Parameters: "-NoProfile -ExecutionPolicy Bypass -File ""{app}\shopdb-admin.ps1"" restart"; \ + Parameters: "-NoProfile -ExecutionPolicy Bypass -File ""{app}\shopdb-admin.ps1"" restart -AppRoot ""{app}"" -SitePort {code:SitePortValue}"; \ WorkingDir: "{app}"; IconFilename: "{app}\shopdb.ico"; \ Comment: "Recycle the application pool" Name: "{group}\Back up the database"; Filename: "powershell.exe"; \ - Parameters: "-NoExit -NoProfile -ExecutionPolicy Bypass -File ""{app}\shopdb-admin.ps1"" backup"; \ + Parameters: "-NoExit -NoProfile -ExecutionPolicy Bypass -File ""{app}\shopdb-admin.ps1"" backup -AppRoot ""{app}"" -SitePort {code:SitePortValue}"; \ WorkingDir: "{app}"; IconFilename: "{app}\shopdb.ico"; \ Comment: "Write a .sql dump to ProgramData" @@ -155,7 +168,7 @@ Name: "{group}\Back up the database"; Filename: "powershell.exe"; \ ; System32 from a 32-bit process. The script also self-corrects, but launching ; it correctly avoids a visible relaunch. Filename: "{win}\Sysnative\WindowsPowerShell\v1.0\powershell.exe"; \ - Parameters: "-NoExit -NoProfile -ExecutionPolicy Bypass -File ""{app}\shopdb-admin.ps1"""; \ + Parameters: "-NoExit -NoProfile -ExecutionPolicy Bypass -File ""{app}\shopdb-admin.ps1"" -AppRoot ""{app}"" -SitePort {code:SitePortValue}"; \ Description: "Open the ShopDB-Flask console"; \ Flags: postinstall skipifsilent unchecked @@ -171,6 +184,12 @@ var BannerSub: TNewStaticText; DetailPanel: TPanel; FooterText: TNewStaticText; + RecheckButton: TNewButton; + // Set by the preflight render. The wizard REFUSES to leave the results page + // while this is True: the check used to be advisory, so a server missing IIS + // sailed through every page and failed in the middle of installing, having + // already put Python on the box. + PreflightBlocked: Boolean; PreflightDone: Boolean; // Shown while the preflight runs. Without it, clicking Next appears to hang: // the check takes a few seconds, runs hidden, and gives no sign of life. @@ -268,6 +287,11 @@ begin Result.Bitmap.LoadFromFile(ExpandConstant('{tmp}\') + Swatch); end; +// Defined further down, next to the preflight rendering they drive. Declared here +// because InitializeWizard wires RecheckClick to a button before that point. +procedure RunPreflight; forward; +procedure RecheckClick(Sender: TObject); forward; + procedure InitializeWizard; var I: Integer; @@ -324,10 +348,22 @@ begin FooterText.Parent := PreflightPage.Surface; FooterText.Left := 0; FooterText.Top := PreflightPage.SurfaceHeight - ScaleY(16); - FooterText.Width := PreflightPage.SurfaceWidth; + FooterText.Width := PreflightPage.SurfaceWidth - ScaleX(84); FooterText.AutoSize := False; FooterText.Caption := ''; + // A blocking page needs a way forward that is not "cancel the installer". The + // operator fixes what the page told them to fix - installs the IIS role, frees + // the port - and re-checks without starting over. + RecheckButton := TNewButton.Create(WizardForm); + RecheckButton.Parent := PreflightPage.Surface; + RecheckButton.Width := ScaleX(78); + RecheckButton.Height := ScaleY(23); + RecheckButton.Left := PreflightPage.SurfaceWidth - RecheckButton.Width; + RecheckButton.Top := PreflightPage.SurfaceHeight - ScaleY(23); + RecheckButton.Caption := 'Check again'; + RecheckButton.OnClick := @RecheckClick; + CheckingPage := CreateOutputProgressPage('Checking this server', 'Reading the current configuration. Nothing is being changed.'); @@ -664,6 +700,94 @@ begin FooterText.Caption := IntToStr(Passes) + ' checks passed. Nothing has been changed on this server.'; end; +// Run the read-only check and paint its results. Called from Next on the welcome +// page, and again whenever the operator fixes something and presses Check again. +// +// Stage 1 changes nothing, so running it before the operator commits to anything +// is free, and it catches the blockers that are expensive later: IIS absent, +// locked config sections, the port in use, MySQL 5.6 missing its index flags. +procedure RunPreflight; +var + I, ResultCode: Integer; + Output: TExecOutput; + PreflightScript, Detail: String; + HasBlockers: Boolean; +begin + // The bundle is NOT extracted yet at wizard time - [Files] is processed during + // the install step. Pull just this one script out of the compressed payload. + ExtractTemporaryFile('shopdb-preflight.ps1'); + PreflightScript := ExpandConstant('{tmp}\shopdb-preflight.ps1'); + + // Visible feedback for the few seconds the check takes. SW_HIDE means the + // operator sees nothing at all otherwise, and a frozen wizard reads as a + // crash. ProgressBar.Style := npbstMarquee because the duration is unknown - + // a bar that sits at 0% is worse than no bar. + CheckingPage.SetText('Looking at IIS, disk, ports and the database...', ''); + CheckingPage.SetProgress(0, 0); + CheckingPage.ProgressBar.Style := npbstMarquee; + CheckingPage.Show; + try + // ExecAndCaptureOutput (6.4.0+) hands back stdout and stderr as string arrays, + // so output never touches disk. That deletes BOTH bugs this page shipped with: + // - Exec() has no shell, so "> file" went to PowerShell as a literal + // argument and no file was ever written (page came up empty); + // - PowerShell 5.1 writes UTF-16LE, and LoadStringFromFile reads bytes as + // ANSI, so the page rendered as garbage with a null between characters. + // Neither failure mode can recur now: there is no file and no encoding step. + // 64-bit PowerShell - see PowerShellPath. The IIS checks fail without it. + ExecAndCaptureOutput(PowerShellPath, + '-NoProfile -ExecutionPolicy Bypass -File "' + PreflightScript + '" -Delimited', + '', SW_HIDE, ewWaitUntilTerminated, ResultCode, Output); + + // Every run repaints from scratch. The old code rendered ONCE and latched, + // so a re-check could not have shown a different answer even if the operator + // had fixed everything. + while DetailPanel.ControlCount > 0 do + DetailPanel.Controls[0].Free; + + PreflightDone := True; + if (not Output.Error) and (GetArrayLength(Output.StdOut) > 0) then + begin + RenderPreflight(Output.StdOut, HasBlockers); + PreflightBlocked := HasBlockers; + end + else + begin + // stderr is captured SEPARATELY, so a failure can report what actually + // went wrong instead of only an exit code. + Detail := ''; + for I := 0 to GetArrayLength(Output.StdErr) - 1 do + Detail := Detail + Output.StdErr[I] + ' '; + if Trim(Detail) = '' then Detail := 'No error output was produced.'; + SetSwatch(BannerBar, 'swatch-warn.bmp'); + BannerText.Caption := 'Could not check this server'; + BannerSub.Caption := 'Nothing has been verified. Continuing is a risk.'; + AddRow(ScaleY(4), 'swatch-warn.bmp', + 'Check did not run (exit ' + IntToStr(ResultCode) + ')', Detail); + FooterText.Caption := 'Nothing has been changed on this server.'; + // A check that could not RUN is not a check that PASSED. It does not block - + // there is no evidence of a problem - but it must not read like a green light. + PreflightBlocked := False; + end; + finally + CheckingPage.Hide; + end; +end; + +procedure RecheckClick(Sender: TObject); +begin + RunPreflight; +end; + +// The port the operator chose, for the [Icons]/[Run] entries. A subpath install +// is reached on the parent site's port, and the console works that out from +// MOUNT_PATH in .env, so the value only has to be right for the own-site case. +function SitePortValue(Param: String): String; +begin + Result := SitePage.Values[1]; + if Trim(Result) = '' then Result := '8090'; +end; + // Where an EXISTING install would be, for the pre-fill reads. // // Not WizardDirValue() alone: these pages are inserted after wpWelcome and so @@ -681,75 +805,26 @@ begin end; function NextButtonClick(CurPageID: Integer): Boolean; -var - ResultCode, I: Integer; - Output: TExecOutput; - PreflightScript, Detail: String; - HasBlockers: Boolean; begin Result := True; if CurPageID = wpWelcome then + RunPreflight; + + // The results page BLOCKS while anything is failing. It used to be advisory: + // the operator read "IIS is not installed", pressed Next, answered five more + // pages, and the install then died partway through with Python already on the + // box. Refusing here costs them nothing - the server is untouched at this + // point - and the page already says what to do about each failure. + if CurPageID = PreflightPage.ID then begin - // Stage 1 is read-only, so running it before the operator commits to - // anything is free and catches the common blockers (IIS absent, locked - // config sections, port in use, MySQL 5.6 missing its index flags). - // The bundle is NOT extracted yet at wizard time - [Files] is processed during - // the install step. Pull just this one script out of the compressed payload. - ExtractTemporaryFile('shopdb-preflight.ps1'); - PreflightScript := ExpandConstant('{tmp}\shopdb-preflight.ps1'); - - // Visible feedback for the few seconds the check takes. SW_HIDE means the - // operator sees nothing at all otherwise, and a frozen wizard reads as a - // crash. ProgressBar.Style := npbstMarquee because the duration is unknown - - // a bar that sits at 0% is worse than no bar. - CheckingPage.SetText('Looking at IIS, disk, ports and the database...', ''); - CheckingPage.SetProgress(0, 0); - CheckingPage.ProgressBar.Style := npbstMarquee; - CheckingPage.Show; - try - - // ExecAndCaptureOutput (6.4.0+) hands back stdout and stderr as string arrays, - // so output never touches disk. That deletes BOTH bugs this page shipped with: - // - Exec() has no shell, so "> file" went to PowerShell as a literal - // argument and no file was ever written (page came up empty); - // - PowerShell 5.1 writes UTF-16LE, and LoadStringFromFile reads bytes as - // ANSI, so the page rendered as garbage with a null between characters. - // Neither failure mode can recur now: there is no file and no encoding step. - // 64-bit PowerShell - see PowerShellPath. The IIS checks fail without it. - ExecAndCaptureOutput(PowerShellPath, - '-NoProfile -ExecutionPolicy Bypass -File "' + PreflightScript + '" -Delimited', - '', SW_HIDE, ewWaitUntilTerminated, ResultCode, Output); - // Build the rows once. Going Back and Next again must not stack a second set - // of controls on top of the first. - if not PreflightDone then + if PreflightBlocked then begin - PreflightDone := True; - if (not Output.Error) and (GetArrayLength(Output.StdOut) > 0) then - begin - RenderPreflight(Output.StdOut, HasBlockers); - if HasBlockers then - MsgBox('This server is not ready. The problems are listed on the next ' - + 'page, each with what to do about it.', mbError, MB_OK); - end - else - begin - // stderr is captured SEPARATELY, so a failure can report what actually - // went wrong instead of only an exit code. - Detail := ''; - for I := 0 to GetArrayLength(Output.StdErr) - 1 do - Detail := Detail + Output.StdErr[I] + ' '; - if Trim(Detail) = '' then Detail := 'No error output was produced.'; - SetSwatch(BannerBar, 'swatch-warn.bmp'); - BannerText.Caption := 'Could not check this server'; - BannerSub.Caption := 'Installation can continue, but nothing has been verified.'; - AddRow(ScaleY(4), 'swatch-warn.bmp', - 'Check did not run (exit ' + IntToStr(ResultCode) + ')', Detail); - FooterText.Caption := 'Nothing has been changed on this server.'; - end; - end; - finally - CheckingPage.Hide; + MsgBox('This server is not ready yet.' + #13#10#13#10 + + 'Each problem above says what to do about it. Fix them, then choose ' + + '"Check again".' + #13#10#13#10 + + 'Nothing has been changed on this server.', mbError, MB_OK); + Result := False; end; end; @@ -992,8 +1067,19 @@ begin else Result := 'The installation could not be completed (exit ' + IntToStr(ResultCode) + ').'; + // NOT "nothing was left running". The stages run with -OnFailure never, so + // nothing is rolled back: whatever had been created by the failing point is + // still there. Claiming otherwise sent operators away believing the server + // was clean when it was half-configured, and the next thing they did was + // install again on top of it. Result := Result + #13#10#13#10 - + 'Nothing was left running. Full details are in:' + #13#10 + + 'This server has been part-configured. Whatever had been done ' + + 'before the failure is still in place.' + #13#10#13#10 + + 'What to do:' + #13#10 + + ' - Fix the cause above, then run this installer again. Re-running ' + + 'is safe and picks up where it left off.' + #13#10 + + ' - Or remove it entirely from Settings > Apps.' + #13#10#13#10 + + 'Full details, including everything that was created, are in:' + #13#10 + ExpandConstant('{log}'); end; end; @@ -1021,14 +1107,36 @@ begin // "Unknown preprocessor directive" before Pascal parsing happens. Keep the // concatenation operator at the start of continuation lines instead. if Failure <> '' then - MsgBox(Failure, mbCriticalError, MB_OK) + begin + MsgBox(Failure, mbCriticalError, MB_OK); + // The final page is headed "ShopDB-Flask is ready" from [Messages]. After a + // failed install that is the last thing the operator reads, and it + // contradicts the error box they just dismissed. Setup cannot be made to + // fail from here, so at least stop it claiming success. + WizardForm.FinishedHeadingLabel.Caption := 'ShopDB-Flask is NOT installed'; + WizardForm.FinishedLabel.Caption := + 'The installation did not complete. This server has been part-configured.' + + #13#10#13#10 + + 'Fix the problem reported above and run this installer again - re-running ' + + 'is safe. Or remove it from Settings > Apps.' + + #13#10#13#10 + + 'The log is at ' + ExpandConstant('{log}'); + end else + begin + // Recorded so the Start Menu shortcut and the console open the address this + // install actually serves, rather than a compile-time guess. + SaveStringToFile(ExpandConstant('{app}\.installed-url'), FinalUrl, False); MsgBox('ShopDB-Flask is installed.' + #13#10#13#10 + 'Open ' + FinalUrl + #13#10#13#10 + 'With no user in the database that page offers to create the first ' - + 'administrator and then runs the setup wizard.', + + 'administrator and then runs the setup wizard.' + + #13#10#13#10 + + 'Day-to-day: use the ShopDB-Flask Console in the Start Menu, or read ' + + ExpandConstant('{app}\docs\OPERATE-WINDOWS.md'), mbInformation, MB_OK); + end; end; end; diff --git a/deploy/windows/installer/build-installer.ps1 b/deploy/windows/installer/build-installer.ps1 index ccef0db..5a0f672 100644 --- a/deploy/windows/installer/build-installer.ps1 +++ b/deploy/windows/installer/build-installer.ps1 @@ -175,6 +175,19 @@ Step 'Generating SBOM' -o (Join-Path $AppOut 'sbom.cdx.json') | ForEach-Object { Say " $_" 'White' } if ($LASTEXITCODE -ne 0) { Die 'SBOM generation failed' } +# Docs the running site serves, plus the runbooks an air-gapped server has no +# other way to reach. Without openapi.json and llms.txt the self-hosted /api/docs +# page is broken on every installed server. +Step 'Staging docs' +$docsOut = Join-Path $AppOut 'docs' +New-Item -ItemType Directory -Path $docsOut -Force | Out-Null +foreach ($doc in @('openapi.json', 'llms.txt', 'api-inventory.json', + 'INSTALL-WINDOWS.md', 'OPERATE-WINDOWS.md', + 'BACKUP-RESTORE.md', 'UPGRADE.md')) { + $src = Join-Path $RepoRoot ('docs\' + $doc) + if (Test-Path $src) { Copy-Item $src $docsOut -Force; Say " $doc" } +} + # Stage the profile INTO the tree: `flask plugin apply-profile` at provisioning # reads the same profile the tree was staged from, so the installed plugin set # and the shipped plugin code cannot drift. diff --git a/deploy/windows/installer/bundle-lock.ps1 b/deploy/windows/installer/bundle-lock.ps1 index a1e862f..7cb317e 100644 --- a/deploy/windows/installer/bundle-lock.ps1 +++ b/deploy/windows/installer/bundle-lock.ps1 @@ -49,6 +49,7 @@ $script:BundlePayloads = @( @{ Name = 'python'; Required = $true; What = 'the Python installer' } @{ Name = 'httpplatformhandler'; Required = $true; What = 'the IIS module that launches waitress' } @{ Name = 'urlrewrite'; Required = $false; What = 'IIS URL Rewrite, for the client-IP rule' } + @{ Name = 'mysqlclient'; Required = $false; What = 'mysql/mysqldump, for backups against a remote database' } @{ Name = 'mysql'; Required = $false; What = 'MySQL, for the bundled-database option' } ) diff --git a/deploy/windows/installer/shopdb-admin.ps1 b/deploy/windows/installer/shopdb-admin.ps1 index 03d783b..c616fea 100644 --- a/deploy/windows/installer/shopdb-admin.ps1 +++ b/deploy/windows/installer/shopdb-admin.ps1 @@ -25,6 +25,11 @@ param( 'check','sessions','plugins','add-plugin','verify','uninstall')] [string] $Command = 'menu', [string] $Path = '', + # Machine-readable output for 'check'. The people running this are expected to + # ask an AI assistant for help, and pasting a screenshot of a console into a + # chat window loses most of what matters. One structured blob they can paste + # gives the assistant real state to reason about instead of guesses. + [switch] $Json, [string] $AppRoot = 'C:\shopdb-flask', [string] $SiteName = 'shopdb-flask', [string] $AppPool = 'shopdbflask', @@ -46,13 +51,27 @@ $AppCmd = Join-Path $env:windir 'System32\inetsrv\appcmd.exe' # than fix every launcher, relaunch under the native PowerShell. 'Sysnative' is # the alias that lets a 32-bit process reach the real System32, and it exists # ONLY for 32-bit processes - hence the guard. + +# Every relaunch below must carry the ORIGINAL arguments through. They used to be +# dropped, so a console started from the Start Menu with -AppRoot D:\shopdb +# relaunched itself with the C:\shopdb-flask default and reported a healthy +# install as missing - and the operator had done nothing wrong. +function Get-ForwardedArgs { + $forward = @('-NoProfile', '-ExecutionPolicy', 'Bypass', + '-File', ('"' + $PSCommandPath + '"'), $Command) + if ($Path) { $forward += @('-Path', ('"' + $Path + '"')) } + if ($AppRoot) { $forward += @('-AppRoot', ('"' + $AppRoot + '"')) } + if ($SiteName) { $forward += @('-SiteName', ('"' + $SiteName + '"')) } + if ($AppPool) { $forward += @('-AppPool', ('"' + $AppPool + '"')) } + if ($SitePort) { $forward += @('-SitePort', $SitePort) } + if ($Json) { $forward += '-Json' } + return $forward +} + if ([Environment]::Is64BitOperatingSystem -and -not [Environment]::Is64BitProcess) { $native = Join-Path $env:windir 'Sysnative\WindowsPowerShell\v1.0\powershell.exe' if (Test-Path $native) { - $relaunch = @('-NoProfile', '-ExecutionPolicy', 'Bypass', - '-File', ('"' + $PSCommandPath + '"'), $Command) - if ($Path) { $relaunch += @('-Path', ('"' + $Path + '"')) } - Start-Process -FilePath $native -ArgumentList $relaunch -Wait -NoNewWindow + Start-Process -FilePath $native -ArgumentList (Get-ForwardedArgs) -Wait -NoNewWindow return } } @@ -65,13 +84,14 @@ if ([Environment]::Is64BitOperatingSystem -and -not [Environment]::Is64BitProces # instead of reporting a false state. $identity = [Security.Principal.WindowsIdentity]::GetCurrent() $principal = New-Object Security.Principal.WindowsPrincipal($identity) -if (-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) { +# 'open' just launches a browser. Prompting for administrator to do that trains +# people to click through UAC, and the Start Menu shortcut uses this command. +if ($Command -ne 'open' -and + -not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) { Write-Host '' Write-Host ' Administrator rights are needed to read IIS state.' -ForegroundColor Yellow Write-Host ' Re-launching elevated - approve the prompt.' -ForegroundColor Yellow - $argList = @('-NoExit', '-NoProfile', '-ExecutionPolicy', 'Bypass', - '-File', ('"' + $PSCommandPath + '"'), $Command) - if ($Path) { $argList += @('-Path', ('"' + $Path + '"')) } + $argList = @('-NoExit') + (Get-ForwardedArgs) try { Start-Process -FilePath 'powershell.exe' -ArgumentList $argList -Verb RunAs | Out-Null } catch { @@ -326,6 +346,14 @@ function Show-Logs { } function Open-Site { + # The installer records the address it actually published at. Prefer it: this + # command runs unelevated, and .env is ACL'd, so working the address out from + # MOUNT_PATH and IIS may not be possible from here. + $recorded = Join-Path $AppRoot '.installed-url' + if (Test-Path $recorded) { + $u = (Get-Content $recorded -TotalCount 1).Trim() + if ($u) { Start-Process $u; Say " opened $u" 'Green'; return } + } $deploy = Get-Deployment if ($deploy.Subpath) { if ($deploy.Port -eq 80) { $u = "http://{0}/{1}/login" -f $env:COMPUTERNAME, $deploy.Alias } @@ -334,7 +362,7 @@ function Open-Site { $u = "http://{0}:{1}/login" -f $env:COMPUTERNAME, $SitePort } Start-Process $u - Say ' opened in your browser' 'Green' + Say " opened $u" 'Green' } function Backup-Db { @@ -385,7 +413,90 @@ function Backup-Db { } } +function Get-CheckState { + <# + Everything an outside reader needs to reason about this server, gathered + without changing anything. Secrets are NEVER included: the database password + lives in DATABASE_URL and this reports the host, port, name and user only. + #> + $deploy = Get-Deployment + $db = Get-DbParts + $state = [ordered]@{ + collected = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ') + computername = $env:COMPUTERNAME + approot = $AppRoot + installed = (Test-Path (Join-Path $AppRoot 'shopdb\__init__.py')) + version = '' + publishedas = if ($deploy.Subpath) { "subpath /$($deploy.Alias) under '$($deploy.Parent)'" } else { "own site '$SiteName' on port $SitePort" } + baseurl = $deploy.BaseUrl + apppool = 'unknown' + poolstate = 'unknown' + sitestate = 'unknown' + responding = $false + httpstatus = 0 + database = $null + pythonversion = '' + plugins = @() + sbomcomponents = 0 + errors = @() + } + foreach ($pair in @(@('version', '.installed-version'))) { + $f = Join-Path $AppRoot $pair[1] + if (Test-Path $f) { $state[$pair[0]] = (Get-Content $f -TotalCount 1).Trim() } + } + $state.apppool = $AppPool + try { + Import-Module WebAdministration -ErrorAction Stop + if (Test-Path "IIS:\AppPools\$AppPool") { $state.poolstate = (Get-Item "IIS:\AppPools\$AppPool").State.ToString() } + if ($deploy.Subpath) { + $app = Get-WebApplication -Site $deploy.Parent -Name $deploy.Alias -ErrorAction SilentlyContinue + $state.sitestate = if ($app) { 'application present' } else { 'application MISSING' } + } else { + $s = Get-Website -Name $SiteName -ErrorAction SilentlyContinue + $state.sitestate = if ($s) { $s.State.ToString() } else { 'site MISSING' } + } + } catch { $state.errors += "IIS: $($_.Exception.Message)" } + + try { + $r = Invoke-WebRequest -Uri ($deploy.BaseUrl + '/') -UseBasicParsing -TimeoutSec 20 + $state.responding = $true + $state.httpstatus = [int] $r.StatusCode + } catch { $state.errors += "HTTP: $($_.Exception.Message)" } + + if ($db) { + $state.database = [ordered]@{ host = $db.Host; port = $db.Port; name = $db.Name; user = $db.User; reachable = $false } + $mysql = Find-MysqlClient + if ($mysql) { + $q = "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema='$($db.Name)';" + $n = $q | & $mysql "-u$($db.User)" "-p$($db.Pass)" "-h$($db.Host)" "-P$($db.Port)" -N 2>$null + if ($LASTEXITCODE -eq 0) { $state.database.reachable = $true; $state.database.tables = [int] $n } + } else { $state.errors += 'no mysql client found, database not probed' } + } else { $state.errors += 'no .env found' } + + $py = Join-Path $AppRoot 'venv\Scripts\python.exe' + if (Test-Path $py) { $state.pythonversion = (& $py -c "import sys; print('%d.%d.%d' % sys.version_info[:3])" 2>$null | Select-Object -First 1) } + + $dir = Join-Path $AppRoot 'plugins' + if (Test-Path $dir) { + $state.plugins = @(Get-ChildItem $dir -Directory -EA SilentlyContinue | + Where-Object { Test-Path (Join-Path $_.FullName 'manifest.json') } | + Select-Object -ExpandProperty Name) + } + $sbom = Join-Path $AppRoot 'sbom.cdx.json' + if (Test-Path $sbom) { + try { $state.sbomcomponents = (Get-Content $sbom -Raw | ConvertFrom-Json).components.Count } catch { } + } + return $state +} + function Invoke-Check { + if ($Json) { + # ONLY the JSON goes to stdout, so it can be redirected to a file or piped + # without a banner in the middle of the document. + Get-CheckState | ConvertTo-Json -Depth 6 + return + } + Head 'Health check' $flask = Join-Path $AppRoot 'venv\Scripts\flask.exe' if (-not (Test-Path $flask)) { Say ' application not installed' 'Red'; return } @@ -393,6 +504,9 @@ function Invoke-Check { $env:FLASK_APP = 'shopdb' try { & $flask db-utils preflight 2>&1 | ForEach-Object { Say " $_" } } finally { Pop-Location } + Say '' + Say ' For help from an AI assistant, paste the output of:' 'DarkGray' + Say ' shopdb-admin.ps1 check -Json' 'White' } function Show-Sessions { diff --git a/deploy/windows/installer/shopdb-install.ps1 b/deploy/windows/installer/shopdb-install.ps1 index 5c46b35..8125f79 100644 --- a/deploy/windows/installer/shopdb-install.ps1 +++ b/deploy/windows/installer/shopdb-install.ps1 @@ -396,6 +396,19 @@ function Compare-Version { function Find-MysqlTool { param([string] $Name) # mysql.exe or mysqldump.exe + # The BUNDLE first, then the install directory, then the local server. + # + # A site whose MySQL lives on another host has no client installed here at + # all, so mysqldump was never found - which meant every upgrade skipped the + # pre-upgrade backup, and it did so AFTER stage 2 had stopped the pool and + # replaced the application tree. Shipping the client makes the backup work + # on a remote-database site, which is the case that needs it most. + $bundled = @( + (Join-Path $BundleRoot ('mysqlclient\' + $Name)), + (Join-Path $AppRoot ('mysqlclient\' + $Name)) + ) + foreach ($b in $bundled) { if (Test-Path $b) { return $b } } + $roots = @('C:\Program Files\MySQL', 'C:\mysql56\bin', 'C:\Program Files (x86)\MySQL') foreach ($r in $roots) { if (Test-Path $r) { @@ -468,8 +481,14 @@ function Backup-Database { $dump = Find-MysqlTool 'mysqldump.exe' if (-not $dump) { Write-Log 'mysqldump not found; skipping backup' 'WARN'; return '' } + # A dump contains every row, including the users table and its password + # hashes. ProgramData is readable by every user on the box by default, so the + # directory is locked to Administrators and SYSTEM the moment it is created. $dir = Join-Path $env:ProgramData 'ShopDB-Flask\backups' - if (-not (Test-Path $dir)) { New-Item -ItemType Directory -Path $dir -Force | Out-Null } + if (-not (Test-Path $dir)) { + New-Item -ItemType Directory -Path $dir -Force | Out-Null + Protect-File $dir + } $file = Join-Path $dir ("{0}-{1}-{2}.sql" -f $db.Name, $Reason, (Get-Date -Format 'yyyyMMdd-HHmmss')) Write-Log "backing up $($db.Name) before migrating" @@ -749,11 +768,28 @@ FLUSH PRIVILEGES; Write-Log "database $DbName and user $DbUser created" 'OK' Write-Log "app password written to $handoff (stage 2 consumes and deletes it)" - Write-Host '' - Write-Host ' MySQL root password (shown ONCE, not stored anywhere):' -ForegroundColor Yellow - Write-Host (" {0}" -f $rootPass) -ForegroundColor Yellow - Write-Host ' Write it down now. It cannot be recovered.' -ForegroundColor Yellow - Write-Host '' + # NOT Write-Host. Under the wizard this process runs hidden, so nobody ever + # saw this - and every line of stdout is forwarded into the setup log that + # operators are told to send to support. The generated root password was + # therefore invisible to the person who needed it and permanently recorded + # for everyone who did not. + # + # Write it to a file only Administrators and SYSTEM can read, and let the + # wizard tell the operator where it is. + $rootFile = Join-Path $env:ProgramData 'ShopDB-Flask\mysql-root-password.txt' + $rootDir = Split-Path $rootFile -Parent + if (-not (Test-Path $rootDir)) { New-Item -ItemType Directory -Path $rootDir -Force | Out-Null } + New-Item -ItemType File -Force -Path $rootFile | Out-Null + Protect-File $rootFile + Set-Content -Path $rootFile -Value @( + 'MySQL root password for this server, generated during installation.', + 'It is not recorded anywhere else and cannot be recovered.', + 'Move it into your password manager, then delete this file.', + '', + $rootPass + ) -Encoding UTF8 + Write-Log "MySQL root password written to $rootFile (Administrators and SYSTEM only)" 'OK' + Write-Log 'MYSQLROOTFILE:' + $rootFile Write-Log 'stage 0 complete' 'OK' } @@ -900,6 +936,17 @@ a page that cannot load its own assets. Rebuild with scripts/build-site.sh if (-not $WhatIfOnly) { $bundledLock = Join-Path $BundleRoot 'bundle-lock.json' if (Test-Path $bundledLock) { Copy-Item $bundledLock $AppRoot -Force } + # The MySQL client stays on the server. The bundle is extracted to a temp + # directory and deleted when Setup exits, so a copy that lived only there + # would leave `shopdb-admin backup` with nothing to run on a site whose + # database is on another host. + $bundledClient = Join-Path $BundleRoot 'mysqlclient' + if (Test-Path $bundledClient) { + $target = Join-Path $AppRoot 'mysqlclient' + if (-not (Test-Path $target)) { New-Item -ItemType Directory -Path $target -Force | Out-Null } + Copy-Item (Join-Path $bundledClient '*') $target -Recurse -Force + Write-Log 'MySQL client staged for backups' 'OK' + } } foreach ($sub in @('logs','instance')) { $p = Join-Path $AppRoot $sub @@ -1659,15 +1706,44 @@ function Invoke-Stage5 { # requesting http://localhost:8090/ would test a site that does not exist. if ($MountAlias) { $alias = $MountAlias.Trim('/') - $parentPort = 80 + # Prefer an http binding, but fall back to https. A parent site published + # ONLY over https - which is normal, and which the operator did nothing + # wrong to have - left this defaulting to port 80, requesting a URL that + # answers nothing, and failing a working install with a red dialog. + $parentScheme = 'http' + $parentPort = 0 try { - $b = (Get-Website -Name $ParentSite -ErrorAction SilentlyContinue).bindings.Collection | - Where-Object { $_.protocol -eq 'http' } | Select-Object -First 1 + $bindings = (Get-Website -Name $ParentSite -ErrorAction SilentlyContinue).bindings.Collection + $b = $bindings | Where-Object { $_.protocol -eq 'http' } | Select-Object -First 1 + if (-not $b) { + $b = $bindings | Where-Object { $_.protocol -eq 'https' } | Select-Object -First 1 + if ($b) { $parentScheme = 'https' } + } if ($b -and ($b.bindingInformation -match '^[^:]*:(\d+):')) { $parentPort = [int]$Matches[1] } } catch { } - $targets = @("http://localhost:$parentPort/$alias/") + if ($parentPort -eq 0) { $parentPort = if ($parentScheme -eq 'https') { 443 } else { 80 } } + + $targets = @("{0}://localhost:{1}/{2}/" -f $parentScheme, $parentPort, $alias) if ($hostName -and $hostName -ne 'localhost') { - $targets += "http://{0}:{1}/{2}/" -f $hostName, $parentPort, $alias + $targets += "{0}://{1}:{2}/{3}/" -f $parentScheme, $hostName, $parentPort, $alias + } + # An https parent almost certainly has a certificate for its real name, + # not for 'localhost', and a certificate complaint is not an application + # fault. Accept any certificate for the duration of the smoke test only. + if ($parentScheme -eq 'https') { + Write-Log 'parent site is https-only; certificate validation is skipped for this check' 'WARN' + try { + Add-Type -TypeDefinition @' +using System.Net; +using System.Security.Cryptography.X509Certificates; +public class ShopdbSmokeTestCertPolicy : ICertificatePolicy { + public bool CheckValidationResult(ServicePoint sp, X509Certificate cert, WebRequest req, int problem) { + return true; + } +} +'@ -ErrorAction SilentlyContinue + [System.Net.ServicePointManager]::CertificatePolicy = New-Object ShopdbSmokeTestCertPolicy + } catch { } } } else { $targets = @("http://localhost:$SitePort/") @@ -1707,6 +1783,21 @@ function Invoke-Stage5 { } } + # The self-hosted API documentation. It is the thing an operator - or the + # assistant an operator is asking - reaches for on a server with no internet, + # and it is served from a file that has to have been staged into the build. + # A missing file makes /api/docs render an empty page, which nobody notices + # until it is needed. Warn, never fail: the application itself is fine. + $base = $targets[0].TrimEnd('/') + try { + $d = Invoke-WebRequest -Uri "$base/api/docs/openapi.json" -UseBasicParsing -TimeoutSec 20 + if ($d.StatusCode -eq 200) { Write-Log 'API documentation is being served at /api/docs' 'OK' } + else { Write-Log "/api/docs/openapi.json returned $($d.StatusCode)" 'WARN' } + } catch { + Write-Log '/api/docs is NOT working - docs/openapi.json was not staged into this build' 'WARN' + Write-Log ' the site runs fine; offline API reference and LLM assistance will not' 'WARN' + } + # The install is proven working, so the stage-0 handoff is no longer the only # copy of the generated password (.env has it and the app is running on it). # Safe to shred now, and only now - see the note in stage 2. @@ -1806,6 +1897,34 @@ function Invoke-Uninstall { Import-Module WebAdministration -ErrorAction SilentlyContinue + # A SUBPATH install has no site of its own - it is an Application under an + # existing one. Removing only the site left the application in place, pointed + # at a directory this is about to delete, so the parent site (at West + # Jefferson, the live classic ASP) served 503 on that path forever and + # Add/Remove Programs reported success. + # + # The alias is whatever MOUNT_PATH says, which is the same value wsgi.py + # mounts on, so it cannot disagree with how the app was actually published. + $mount = '' + $envFile = Join-Path $AppRoot '.env' + if (Test-Path $envFile) { + $line = Get-Content $envFile -ErrorAction SilentlyContinue | + Where-Object { $_ -like 'MOUNT_PATH=*' } | Select-Object -First 1 + if ($line) { $mount = $line.Substring('MOUNT_PATH='.Length).Trim().Trim('/') } + } + if ($mount) { + $removed = $false + foreach ($site in (Get-Website -ErrorAction SilentlyContinue)) { + $app = Get-WebApplication -Site $site.Name -Name $mount -ErrorAction SilentlyContinue + if ($app) { + Remove-WebApplication -Site $site.Name -Name $mount -ErrorAction SilentlyContinue + Write-Log "removed application /$mount from site '$($site.Name)'" 'OK' + $removed = $true + } + } + if (-not $removed) { Write-Log "application /$mount not present" } + } + if (Get-Website -Name $SiteName -ErrorAction SilentlyContinue) { Remove-Website -Name $SiteName -ErrorAction SilentlyContinue Write-Log "removed site $SiteName" 'OK' @@ -1820,11 +1939,22 @@ function Invoke-Uninstall { Write-Log "removed app pool $AppPool" 'OK' } else { Write-Log "app pool $AppPool not present" } - $rule = "ShopDB-Flask $SitePort" + # Stage 4 creates the rule as "$SiteName $SitePort". This looked for the + # literal "ShopDB-Flask 8090", which is not the same string as the default + # "shopdb-flask 8090" and matches nothing at all on a non-default port, so + # the rule outlived the uninstall. Build the name the same way stage 4 does, + # and sweep any rule left by a differently-ported install of the same site. + $rule = "$SiteName $SitePort" if (Get-NetFirewallRule -DisplayName $rule -ErrorAction SilentlyContinue) { Remove-NetFirewallRule -DisplayName $rule -ErrorAction SilentlyContinue Write-Log "removed firewall rule '$rule'" 'OK' } else { Write-Log "firewall rule '$rule' not present" } + Get-NetFirewallRule -ErrorAction SilentlyContinue | + Where-Object { $_.DisplayName -like "$SiteName *" -and $_.DisplayName -ne $rule } | + ForEach-Object { + Remove-NetFirewallRule -DisplayName $_.DisplayName -ErrorAction SilentlyContinue + Write-Log "removed leftover firewall rule '$($_.DisplayName)'" 'OK' + } if (Test-Path $AppRoot) { # .env holds the database password in plaintext by design (the app reads it diff --git a/deploy/windows/installer/shopdb-preflight.ps1 b/deploy/windows/installer/shopdb-preflight.ps1 index 3e8285b..dfaaeee 100644 --- a/deploy/windows/installer/shopdb-preflight.ps1 +++ b/deploy/windows/installer/shopdb-preflight.ps1 @@ -368,6 +368,31 @@ Invoke-Check 'MySQL' 'Port 3306' { } } +Invoke-Check 'MySQL' 'Backup client' { + # mysqldump is what takes the mandatory pre-upgrade backup. Without it every + # upgrade skips the backup - and skips it AFTER the application pool has been + # stopped and the tree replaced, so the site is down and there is nothing to + # restore from. A site whose database is on another server typically has no + # MySQL client installed here at all, which is exactly the case that needs it. + $names = @('mysqldump.exe') + $found = '' + foreach ($root in @((Join-Path $PSScriptRoot 'mysqlclient'), + (Join-Path $AppRoot 'mysqlclient'), + 'C:\Program Files\MySQL', 'C:\mysql56\bin', + 'C:\Program Files (x86)\MySQL')) { + if (-not (Test-Path $root)) { continue } + $hit = Get-ChildItem $root -Filter $names[0] -Recurse -ErrorAction SilentlyContinue | + Select-Object -First 1 + if ($hit) { $found = $hit.FullName; break } + } + if ($found) { + Add-Result 'MySQL' 'Backup client' 'PASS' "mysqldump found ($found)" + } else { + Add-Result 'MySQL' 'Backup client' 'WARN' 'mysqldump not found on this server' ` + 'Needed for the automatic pre-upgrade backup and for "shopdb-admin.ps1 backup". A first install works without it; upgrades will not be protected. Add mysqlclient\ to the installer bundle, or install the MySQL client on this server.' + } +} + Invoke-Check 'MySQL' 'Version and config' { # Find mysqld.exe via the service binary path; read the version and locate my.ini. $svc = Get-CimInstance Win32_Service -ErrorAction SilentlyContinue | diff --git a/deploy/windows/installer/verify_bundle_lock.py b/deploy/windows/installer/verify_bundle_lock.py index ad8fc2b..80634f0 100644 --- a/deploy/windows/installer/verify_bundle_lock.py +++ b/deploy/windows/installer/verify_bundle_lock.py @@ -31,6 +31,7 @@ PAYLOADS = [ ('python', True, 'the Python installer'), ('httpplatformhandler', True, 'the IIS module that launches waitress'), ('urlrewrite', False, 'IIS URL Rewrite, for the client-IP rule'), + ('mysqlclient', False, 'mysql/mysqldump, for backups against a remote database'), ('mysql', False, 'MySQL, for the bundled-database option'), ] diff --git a/docs/BACKUP-RESTORE.md b/docs/BACKUP-RESTORE.md index 45f171f..197a16e 100644 --- a/docs/BACKUP-RESTORE.md +++ b/docs/BACKUP-RESTORE.md @@ -142,8 +142,40 @@ version both are no-ops. - `curl -s -X POST -H "Content-Type: application/json" -d '{}' http://localhost:5001/api/auth/login | jq .` should return a `VALIDATION_ERROR`, not a 500. +## Windows sites (installer-built) + +On a server installed from the Windows installer, everything above is wrapped by +the operator console. Do not run mysqldump by hand: + +```powershell +cd C:\shopdb-flask +.\shopdb-admin.ps1 backup # C:\ProgramData\ShopDB-Flask\backups +.\shopdb-admin.ps1 backup D:\backups +``` + +The dump is verified complete before it is reported as good; a truncated one is +deleted rather than left to be discovered when it is needed. An upgrade takes its +own backup automatically before touching the schema, and restores from it if a +migration fails. + +Two Windows-specific notes: + +- The backup directory is locked to Administrators and SYSTEM, because a dump + contains every row including user password hashes. Keep it that way. +- `mysqldump` must be present. It ships with the bundled-database option; a site + using a remote MySQL needs `mysqlclient\` in its installer bundle, or the + pre-upgrade backup is skipped. `shopdb-admin.ps1 check` reports this. + +Restoring is the standard `mysql < dump.sql`, then +`.\shopdb-admin.ps1 restart`. Also restore `C:\shopdb-flask\instance\` if you are +rebuilding a server - it holds uploaded branding and map blueprints, which the +database does not. + +See [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md). + ## See also - [DEPLOY.md](DEPLOY.md) - first-time deploy +- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - Windows Server install - [UPGRADE.md](UPGRADE.md) - upgrade procedure (back up first) - [CONFIG.md](CONFIG.md) - environment variables and Setting keys diff --git a/docs/DEPLOY-WINDOWS-IIS.md b/docs/DEPLOY-WINDOWS-IIS.md index 3c36736..ac71dfe 100644 --- a/docs/DEPLOY-WINDOWS-IIS.md +++ b/docs/DEPLOY-WINDOWS-IIS.md @@ -1,5 +1,13 @@ # Deploy shopdb-flask to Windows IIS (MySQL 5.6) +> **Not the route for a new site.** Sister sites install from the Windows +> installer — one `.exe`, no manual IIS work: **[INSTALL-WINDOWS.md](INSTALL-WINDOWS.md)**. +> +> This is the **manual** procedure for the West Jefferson server, which was built +> by hand against its existing MySQL 5.6 and predates the installer. Keep it for +> that box. + + Runbook for standing up a single-site instance on the production Windows Server that already runs the classic ASP shopdb, using IIS + HttpPlatformHandler + waitress, against the existing MySQL 5.6. This is the test-instance path; keep diff --git a/docs/INSTALL-WINDOWS-IIS.md b/docs/INSTALL-WINDOWS-IIS.md index 15c78a0..8ba14da 100644 --- a/docs/INSTALL-WINDOWS-IIS.md +++ b/docs/INSTALL-WINDOWS-IIS.md @@ -1,5 +1,13 @@ # ShopDB - Windows + IIS install runbook +> **Not the route for a new site.** Sister sites install from the Windows +> installer — one `.exe`, no manual IIS work: **[INSTALL-WINDOWS.md](INSTALL-WINDOWS.md)**. +> +> This document is the **manual** procedure, kept for reference and for +> hand-built servers that predate the installer. Note that the installer will not +> adopt a server built this way without `-AdoptExisting`, on purpose. + + A step-by-step, **tested** install for a new site on Windows Server / Windows 11 with IIS in front of the Flask app (HttpPlatformHandler -> waitress), backed by MySQL. This runbook was validated end to end on a win11 + IIS + MySQL 5.6 box. diff --git a/docs/INSTALL-WINDOWS.md b/docs/INSTALL-WINDOWS.md new file mode 100644 index 0000000..0005980 --- /dev/null +++ b/docs/INSTALL-WINDOWS.md @@ -0,0 +1,201 @@ +# Install ShopDB-Flask on Windows Server + +**This is the route for a new site.** You run one `.exe`, answer a few questions, +and get a working application. Nothing here needs an internet connection, and you +do not need to know IIS, Python or MySQL. + +If you are looking after an existing hand-built server, see +[DEPLOY-WINDOWS-IIS.md](DEPLOY-WINDOWS-IIS.md) instead — that is the manual +procedure, and the installer will not adopt a server it did not build. + +--- + +## Before you start + +You need **four things**. The installer supplies everything else. + +| | What | How to check | +|---|---|---| +| 1 | Windows Server 2019 or newer | `winver` | +| 2 | The **IIS Web Server role** installed | Server Manager → Manage → Add Roles → Web Server (IIS). Or run the PowerShell below. | +| 3 | Administrator rights on the box | Right-click PowerShell → "Run as administrator" works | +| 4 | A decision about the database — see [Which database?](#which-database) | — | + +Installing IIS, if it is missing (this needs no internet): + +```powershell +Install-WindowsFeature -Name Web-Server -IncludeManagementTools +``` + +The installer **checks all of this before it changes anything**, and it will not +let you continue until the check passes. You do not have to get it right first +time. + +### Which database? + +Two options. Pick before you start, because they ask different questions. + +- **Use the bundled MySQL** — the installer puts MySQL 8.0 on this server and + creates the database for you. Choose this when the server has no database + today. Simplest option, nothing to arrange in advance. +- **Use an existing MySQL** — the database already exists somewhere, and you have + a hostname, a database name, a username and a password for it. Choose this if + your site already runs MySQL, or a DBA looks after it. + +If you are unsure: if nobody has given you database credentials, you want the +bundled option. + +--- + +## Installing + +1. Copy the installer `.exe` onto the server. It is one file and needs no + network. +2. **Right-click it → Run as administrator.** Without this it cannot configure + IIS, and it will tell you so. +3. Work through the wizard. The pages are: + +| Page | What it wants | If unsure | +|---|---|---| +| **Server check** | Nothing — it reports what it found | Fix anything red, then "Check again". You cannot continue while something is red, and nothing has been changed yet. | +| **Features** | Which parts of the product this site uses | The defaults are fine. You can add more later; removing needs a new installer. | +| **Database** | Bundled or existing — see above | Bundled | +| **Database details** | Host, port, name, user, password | Only asked for the existing-database option | +| **Address** | How people reach the site | See [Own address or subpath?](#own-address-or-subpath) | +| **Client addresses** | Whether a proxy sits in front | See [Client addresses](#client-addresses) | +| **Location** | Where to install | `C:\shopdb-flask` is fine | + +4. The install takes a few minutes. Most of it is Python and the database schema. +5. At the end you get the address to open. **Write it down** — it is also on the + Start Menu as "Open ShopDB-Flask". + +### Own address or subpath? + +- **Its own address** — `http://yourserver:8090/`. Choose this on a server that + is not already running a website. Simplest. +- **Under this server's existing address** — `http://yourserver/shopdb/`. Choose + this when the server already serves something else and you do not want a second + port or a new DNS name. This is what West Jefferson uses. + +You cannot change your mind later without re-running the installer, because the +web interface has the address compiled into it. + +### Client addresses + +The application records who connects, and some features decide what to show based +on it. The wizard asks one question: + +- **Clients connect to this server directly** — the normal answer. Pick this + unless you know otherwise. +- **A proxy or load balancer sits in front** — pick this only if your network + team has told you traffic reaches this server through something else first. + +Getting this wrong is not dangerous, but the site will record every visitor as +coming from the server itself, and features that depend on location will not +work. It can be changed later by re-running the installer. + +--- + +## First login + +Open the address the installer gave you. With no users in the database yet, the +page offers to **create the first administrator**, then runs a short setup wizard +for site details, features and the floor map. + +That first account is a normal administrator account. Use a real password — +this is the account that creates everyone else. + +--- + +## Did it work? + +From the Start Menu, open **ShopDB-Flask Console** and pick option 1, or: + +```powershell +cd C:\shopdb-flask +.\shopdb-admin.ps1 status +``` + +You want to see the site started, the pool started, and `responding : yes`. + +Day-to-day tasks — restarting, backups, logs, upgrades — are in +[OPERATE-WINDOWS.md](OPERATE-WINDOWS.md). + +--- + +## If the install fails + +**The server is left part-configured.** Whatever had been done before the failure +is still there. That is deliberate: it means re-running is able to pick up where +it stopped. + +1. Read the error. It names the cause and what to do about it. +2. Fix that, then **run the same installer again**. Re-running is safe — it skips + what is already done and does not touch your database or `.env`. +3. If you would rather start clean, remove it from **Settings → Apps** first. + +The full log is at: + +``` +C:\ProgramData\ShopDB-Flask\logs\shopdb-install-.log +``` + +It records every step, including everything that was created. Send this if you +need help. + +### Getting help from an AI assistant + +These installs are often done with an assistant open in another window. Give it +real state rather than a description: + +```powershell +.\shopdb-admin.ps1 check -Json +``` + +That prints one structured block covering the version, how the site is published, +IIS state, database reachability, Python version, installed features and any +errors. Paste it in. **It contains no passwords.** The install log is also safe +to share — the installer keeps secrets out of it deliberately. + +Offline API reference for this server is served at `/api/docs` on the site +itself, and `docs\` in the install directory holds these runbooks. + +--- + +## Upgrading + +Run a newer installer over the top. It: + +- backs the database up first, **verifies the dump is complete**, and refuses to + continue if it cannot; +- restores from that backup if the schema migration fails; +- refuses to install an **older** build over a newer one; +- keeps your `.env`, your data and your `web.config`. + +Nothing else is required. See [UPGRADE.md](UPGRADE.md). + +> **Before your first upgrade:** confirm `mysqldump` is available — the console's +> health check reports it. Without it the pre-upgrade backup is skipped, and that +> is the one you would want if a migration went wrong. It ships with the bundled +> database option; for an existing remote database, ask for `mysqlclient\` to be +> included in your installer bundle. + +--- + +## Removing it + +**Settings → Apps → ShopDB-Flask**, or Add/Remove Programs. + +That removes the website, the application pool, the firewall rule and the +application directory. It deliberately **does not** drop the database and does +not uninstall MySQL, so your data survives. + +Take a backup first: `.\shopdb-admin.ps1 backup` + +--- + +## Notes for the person who builds the installer + +Building a bundle for a site is a separate job, documented in +[../deploy/windows/installer/README.md](../deploy/windows/installer/README.md). +Sites receive a finished `.exe`; they do not build one. diff --git a/docs/OPERATE-WINDOWS.md b/docs/OPERATE-WINDOWS.md new file mode 100644 index 0000000..9e5c3f2 --- /dev/null +++ b/docs/OPERATE-WINDOWS.md @@ -0,0 +1,195 @@ +# Running ShopDB-Flask on Windows Server + +Day-to-day operation of a site installed with the Windows installer. If you are +installing for the first time, start with [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md). + +Everything here goes through one tool, installed alongside the application: + +``` +C:\shopdb-flask\shopdb-admin.ps1 +``` + +The Start Menu folder **ShopDB-Flask** has shortcuts for the common tasks. Run it +with no arguments for a menu, or pass a command directly. It needs +Administrator — it will ask, except for `open`. + +--- + +## The commands + +| Command | What it does | Safe at any time | +|---|---|---| +| `status` | Is it published, running, responding; database and table count | yes | +| `restart` | Recycles the application pool. **Use this after any config change.** | yes — drains requests rather than cutting them off | +| `stop` / `start` | Takes the site down / brings it back | yes, but `stop` makes it unavailable | +| `logs` | Last lines of the application and install logs | yes | +| `check` | Full health check | yes | +| `check -Json` | The same, machine-readable — see [Getting help](#getting-help) | yes | +| `verify` | Which build this is, and whether what is installed still matches it | yes | +| `sessions` | IIS worker processes and memory | yes | +| `plugins` | Which features are installed, which are available | yes | +| `add-plugin -Path ` | Turns on a feature this build ships | changes the site; restarts it | +| `backup [-Path ]` | Writes a verified `.sql` dump | yes, but see below | +| `open` | Opens the site in a browser | yes | + +Examples: + +```powershell +.\shopdb-admin.ps1 status +.\shopdb-admin.ps1 restart +.\shopdb-admin.ps1 backup D:\backups +.\shopdb-admin.ps1 verify -Path leaflet +``` + +--- + +## Backups + +```powershell +.\shopdb-admin.ps1 backup +``` + +Writes to `C:\ProgramData\ShopDB-Flask\backups` unless you pass a directory. The +dump is **verified complete** before it is reported as good — a truncated backup +is deleted rather than left to be discovered later. + +Two things to know: + +- **The dump contains everything, including user password hashes.** The directory + is locked to Administrators and SYSTEM. Keep it that way, and treat copies as + sensitive. +- **Store it off this server.** A backup on the server does not survive the + server. + +An upgrade takes its own backup automatically, before it touches the schema. + +Restore, and the Linux/Docker equivalents, are in +[BACKUP-RESTORE.md](BACKUP-RESTORE.md). + +--- + +## Upgrading + +Run a newer installer over the top. Nothing else. It backs up first, refuses to +go backwards, and restores if a migration fails. See +[UPGRADE.md](UPGRADE.md). + +--- + +## Adding a feature + +```powershell +.\shopdb-admin.ps1 plugins # what is here +.\shopdb-admin.ps1 add-plugin -Path warranty # turn one on +``` + +Only features **shipped in this build** can be added. Each site's installer is +built for that site's chosen feature set, so a feature nobody asked for is not on +the server at all — adding it means a new installer built from an updated +profile. `plugins` shows you which is which. + +--- + +## When something is wrong + +Work down this list. + +**1. Is it actually down?** + +```powershell +.\shopdb-admin.ps1 status +``` + +`responding : NO` with the pool `Started` usually means the application failed to +start, not that IIS is broken. + +**2. What does it say?** + +```powershell +.\shopdb-admin.ps1 logs +``` + +Application logs are in `C:\shopdb-flask\logs`, install logs in +`C:\ProgramData\ShopDB-Flask\logs`. + +**3. Try a restart.** It fixes anything that is a stuck worker, and tells you +immediately if it is not: + +```powershell +.\shopdb-admin.ps1 restart +``` + +**4. Check the database is reachable** — `status` reports the host and whether it +could count tables. A site that starts but shows no data is usually a database +problem, not an application one. + +**5. Confirm nothing has drifted:** + +```powershell +.\shopdb-admin.ps1 verify +``` + +This flags packages that no longer match what shipped — which usually means +somebody ran a `pip install` on the server by hand. + +--- + +## Getting help + +Give an assistant real state rather than describing the symptom: + +```powershell +.\shopdb-admin.ps1 check -Json +``` + +One structured block: version, how the site is published, IIS and pool state, +whether it responds, database host and reachability, Python version, installed +features, and any errors. **It contains no passwords** and is safe to paste into +a chat window or a ticket. + +The install log is also safe to share — secrets are deliberately kept out of it. + +Offline reference on the server itself: + +- `/api/docs` on the site — the full API reference, self-hosted, no internet. +- `C:\shopdb-flask\docs\` — these runbooks. +- `C:\shopdb-flask\sbom.cdx.json` — every component this build contains. + +--- + +## Answering "are we affected by this vulnerability?" + +The server carries its own bill of materials, so this does not need the build box +or an internet connection: + +```powershell +.\shopdb-admin.ps1 verify -Path +``` + +It reports whether the component is here, at what version, and whether it +actually **ships** or is only used to build the software. Example: + +``` +matches for 'leaflet': + leaflet 1.9.4 SHIPPED +``` + +Nothing found means this server does not carry it. + +--- + +## Where things live + +| | | +|---|---| +| Application | `C:\shopdb-flask` | +| Configuration and secrets | `C:\shopdb-flask\.env` (locked down — do not loosen) | +| Application logs | `C:\shopdb-flask\logs` | +| Install logs | `C:\ProgramData\ShopDB-Flask\logs` | +| Backups | `C:\ProgramData\ShopDB-Flask\backups` | +| Bill of materials | `C:\shopdb-flask\sbom.cdx.json` | +| Which build this is | `C:\shopdb-flask\.installed-version` | + +If the bundled MySQL was installed, its generated root password was written once +to `C:\ProgramData\ShopDB-Flask\mysql-root-password.txt`. **Move it into your +password manager and delete that file.** It cannot be recovered. diff --git a/docs/UPGRADE.md b/docs/UPGRADE.md index f27dfc0..5fd0e89 100644 --- a/docs/UPGRADE.md +++ b/docs/UPGRADE.md @@ -111,9 +111,33 @@ print(Setting.query.filter(Setting.key.like('map_blueprint%')).all()) PY ``` +## Windows sites (installer-built) + +Run a newer installer `.exe` over the existing install. That is the whole +procedure - none of the manual steps above apply. + +It backs the database up first and verifies the dump, applies the core and plugin +migrations, restores from that backup if a migration fails, and refuses to +install an older build over a newer one. Your `.env`, your data and your +`web.config` are kept. + +Before the first upgrade, confirm `mysqldump` is available +(`.\shopdb-admin.ps1 check`). Without it the pre-upgrade backup is skipped, which +is the one you would want if a migration went wrong. + +Afterwards: + +```powershell +.\shopdb-admin.ps1 status +.\shopdb-admin.ps1 verify +``` + +See [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md). + ## See also - [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore +- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - Windows Server install - [CONFIG.md](CONFIG.md) - environment variables and Setting keys - [DEPLOY.md](DEPLOY.md) - first-time deploy runbook - `CHANGELOG.md` - what changed in each release diff --git a/docs/llms.txt b/docs/llms.txt index f1def7a..8cc1fbc 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -5,6 +5,29 @@ network devices, measuring tools, applications, knowledge base, USB, warranties) for GE Aerospace sites. This file is the quick entrypoint; the full machine spec is the OpenAPI at `/api/docs/openapi.json` (browse it at `/api/docs`). +## Installing and operating this server (read this before answering "how do I install/fix it") +The people running these servers are often not Windows, IIS or Python +specialists, and they ask assistants for help. Two documents are authoritative: +- `docs/INSTALL-WINDOWS.md` - installing a NEW site. One offline installer `.exe`. +- `docs/OPERATE-WINDOWS.md` - restart, logs, backups, upgrades, troubleshooting. +Both ship in `docs/` inside the install directory on every installed server. + +Do NOT walk someone through `docs/INSTALL-WINDOWS-IIS.md` or +`docs/DEPLOY-WINDOWS-IIS.md` for a new site. Those are the MANUAL procedure, kept +only for hand-built servers that predate the installer; following them produces a +server the installer then refuses to upgrade. + +Day-2 operations all go through `shopdb-admin.ps1` in the install directory +(default `C:\shopdb-flask`): `status`, `restart`, `logs`, `check`, `verify`, +`backup`, `plugins`, `open`. Before diagnosing anything, ask for the output of +`shopdb-admin.ps1 check -Json` - it reports version, publishing method, IIS and +pool state, HTTP reachability, database host and reachability, Python version, +installed plugins and errors, and it contains no secrets. `verify -Path ` +answers "does this server carry component X" from the on-box CycloneDX SBOM. + +Python is 3.14 and the wheelhouse is locked to it; an upgrade against a venv +built by a different minor version is refused by design. + ## Base URL Prod (West Jefferson): `https://tsgwp00525.wjs.geaerospace.net/shopdb` All API paths are under `/api` (e.g. `/api/assets`). Dev: `http://localhost:5001`. diff --git a/scripts/build-site.sh b/scripts/build-site.sh index 7d3b4c1..4fcb40a 100755 --- a/scripts/build-site.sh +++ b/scripts/build-site.sh @@ -86,6 +86,22 @@ fi echo "==> Generating SBOM ..." python3 "$REPO/scripts/generate_sbom.py" "$REPO" -o "$OUT/sbom.cdx.json" +# Docs the RUNNING SITE serves or the operator needs on the box. Without +# openapi.json and llms.txt the self-hosted /api/docs page is broken on every +# installed server - which matters most for the sites least able to debug it, and +# for the operators who will point an assistant at their own instance. The +# Windows runbooks ship because an air-gapped server has no other way to reach +# them; docs/ is not otherwise staged, so shipping the whole tree would be noise. +echo "==> Staging docs ..." +mkdir -p "$OUT/docs" +for doc in openapi.json llms.txt api-inventory.json \ + INSTALL-WINDOWS.md OPERATE-WINDOWS.md BACKUP-RESTORE.md UPGRADE.md; do + [ -f "$REPO/docs/$doc" ] && cp -a "$REPO/docs/$doc" "$OUT/docs/" +done +# Never let a missing optional doc fail the build - the loop's last test governs +# the exit status under `set -e`. +true + # Stage the profile INTO the tree. This is what makes the set self-describing: # `flask plugin apply-profile` at provisioning time reads the same profile the # tree was staged from, so the installed plugin set and the shipped plugin code