Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption import, a versioned agent binary store with upload and rollback, remote-BuildKit CI, tray version reporting, and UI fixes. Applied as a patch again - the published mirror rewrites hashes, so the branches have unrelated histories and git refuses to merge them. Reviewed by three agents before merging. Four things fixed here. THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in publish-images.yml sits at column 0 instead of indented under steps:, so the whole file is invalid YAML and GitHub would refuse it. Their entire GHCR distribution channel - which Start-ShopDBDev.ps1 and the compose files consume - was broken at the source and clearly never run. Indented; all five workflows now parse. THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the approval packet from the same upload and then asked the writer whether they matched, so packet_matches_plan compared a hash against itself. The documented contract - "a human approves a specific, frozen plan, and the writer can prove nothing drifted" - did not exist in the web path: an admin could preview file A and commit file B, gated only by the needs-create group NAMES they ticked. The preview's planhash is now required at commit and checked against a freshly rebuilt plan. Their tests committed without previewing, which is what let this through; they go through the real flow now, and three new tests pin it - all confirmed to fail with the check stubbed out. endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a ZIP"; with the upload routes it also means "choose the executable every future package installs and runs as SYSTEM". Before this, staging needed filesystem access to the server. The permission description still said only "generate and download a package", so an admin granting it could not see what they were granting. Reworded. NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size and timestamp; re-uploading a version replaces the bytes in place under the same label. A sha256 of the stored file is now recorded, and the upload is audit-logged with it - after an incident there was previously no answer to who put that binary on the fleet. Also documented the nine new routes in docs/api-inventory.json (that gate cannot run on the mirror, so it falls to us at promotion time again), regenerated the spec and the project map, and kept our own docs where they conflicted: docs/ is scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file rather than our 737-line canonical one. Their journal section is appended to ours. Contract version comes across as 0.30.0 for a genuinely additive surface. Suite 2369, frontend 324, gate green. Scope: product
379 lines
15 KiB
PowerShell
379 lines
15 KiB
PowerShell
<#
|
|
.SYNOPSIS
|
|
Build a channel-aware ShopDB distributable installer (stable or edge).
|
|
|
|
.DESCRIPTION
|
|
Produces ONE self-contained installer package for the requested channel:
|
|
|
|
stable -> built from 'main', no DEV badge (SHOPDB_ENV_LABEL empty).
|
|
edge -> built from 'main' (the trunk), red DEV badge (SHOPDB_ENV_LABEL=DEV).
|
|
|
|
This is the distributable counterpart to scripts/dev-bootstrap.ps1 (which
|
|
stands a dev box up FROM the repo). Here the operator has no repo: the script
|
|
builds the app image from the channel branch on a connected build box, saves
|
|
it plus MySQL into a loadable tarball (reusing the exact bundle mechanics of
|
|
build-offline-bundle.ps1), and drops a runnable package:
|
|
|
|
dist/shopdb-<channel>-<version>/
|
|
shopdb-stack-<version>.tar.gz # docker load input
|
|
shopdb-stack-<version>.tar.gz.sha256
|
|
docker-compose.airgap.yml # image-only run template
|
|
docker-compose.channel.yml # SHOPDB_ENV_LABEL overlay
|
|
.env # channel-filled, secrets as CHANGE_ME_* sentinels
|
|
Install-ShopDB.ps1 # site-side one-command installer
|
|
README.txt
|
|
|
|
Keeping stable and edge as ONE parameterized builder (not two scripts) is the
|
|
same anti-drift decision as the config-driven badge: the channels differ only
|
|
by branch, image tag, and label - all data, not code.
|
|
|
|
.PARAMETER Channel
|
|
'stable' (the newest release tag) or 'edge' (main, the trunk). Required.
|
|
|
|
.PARAMETER Version
|
|
Image tag / bundle version. Defaults to a channel-prefixed date stamp so two
|
|
channels built the same day never collide (e.g. 'stable-2024.06.01').
|
|
|
|
.PARAMETER Ref
|
|
Override the git ref to build. Defaults to the newest v* tag for stable, 'main' for
|
|
edge. Lets a release engineer cut an installer from a tag or fix branch.
|
|
|
|
.PARAMETER MysqlImage
|
|
MySQL image the stack runs (default mysql:8.4). Must match db.image in
|
|
docker-compose.airgap.yml.
|
|
|
|
.PARAMETER OutDir
|
|
Root for the generated package (default ./dist).
|
|
|
|
.PARAMETER SkipBuild
|
|
Reuse an already-built shopdb-flask:<Version> image; only save + package.
|
|
|
|
.EXAMPLE
|
|
pwsh scripts/build-channel-installer.ps1 -Channel stable
|
|
|
|
.EXAMPLE
|
|
pwsh scripts/build-channel-installer.ps1 -Channel edge -Version edge-preview3
|
|
|
|
.NOTES
|
|
ASCII only, no emoji or smart punctuation, per repo style. PowerShell 5.1+.
|
|
Requires Docker Desktop and (unless -SkipBuild) clean access to build the
|
|
image. Secrets ship as CHANGE_ME_* placeholders and the generated installer
|
|
refuses to start until they are rotated, so a package can never go live on
|
|
shipped credentials.
|
|
#>
|
|
[CmdletBinding()]
|
|
param(
|
|
[Parameter(Mandatory)]
|
|
[ValidateSet('stable', 'edge')]
|
|
[string]$Channel,
|
|
|
|
[string]$Version,
|
|
|
|
[string]$Ref,
|
|
|
|
[string]$MysqlImage = 'mysql:8.4',
|
|
|
|
[string]$OutDir = 'dist',
|
|
|
|
[switch]$SkipBuild
|
|
)
|
|
|
|
$ErrorActionPreference = 'Stop'
|
|
|
|
# Channel -> (ref, badge label). The single source of channel truth, mirrored
|
|
# from dev-bootstrap.ps1 so the two entry points never disagree about what a
|
|
# channel means.
|
|
#
|
|
# stable has NO fixed ref: a release here is a tag, not a branch, so it is
|
|
# resolved below from the newest v* tag. Writing 'main' here would build the
|
|
# trunk and hand it to a site under a word that promises a release - the same
|
|
# mislabel this repo's channel mapping was corrected for, missed in this file
|
|
# the first time round.
|
|
$channels = @{
|
|
stable = @{ Ref = $null; EnvLabel = '' }
|
|
edge = @{ Ref = 'main'; EnvLabel = 'DEV' }
|
|
}
|
|
|
|
function Resolve-StableRef {
|
|
# Newest release tag by version sort. Falls back to main with a warning
|
|
# rather than silently building something and calling it stable.
|
|
$tag = (git tag --list 'v*' --sort='-v:refname' | Select-Object -First 1)
|
|
if ($tag) { return $tag }
|
|
Write-Warning 'No v* tag in this repo; building stable from main.'
|
|
return 'main'
|
|
}
|
|
}
|
|
|
|
function Assert-LastExit([string]$What) {
|
|
if ($LASTEXITCODE -ne 0) { throw "$What failed (exit $LASTEXITCODE)" }
|
|
}
|
|
|
|
if (-not (Get-Command docker -ErrorAction SilentlyContinue)) {
|
|
throw 'docker not found on PATH. Install Docker Desktop and retry.'
|
|
}
|
|
if (-not (Get-Command git -ErrorAction SilentlyContinue)) {
|
|
throw 'git not found on PATH. Install Git and retry.'
|
|
}
|
|
|
|
$config = $channels[$Channel]
|
|
# An explicit -Ref always wins. Otherwise the channel decides: edge is the
|
|
# trunk, stable resolves the newest release tag.
|
|
$branch = if ($Ref) { $Ref }
|
|
elseif ($config.Ref) { $config.Ref }
|
|
else { Resolve-StableRef }
|
|
if (-not $Version) { $Version = "$Channel-$(Get-Date -Format 'yyyy.MM.dd')" }
|
|
|
|
$appImage = "shopdb-flask:$Version"
|
|
$repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..')).Path
|
|
|
|
Write-Host "ShopDB channel installer builder" -ForegroundColor White
|
|
Write-Host " Channel : $Channel" -ForegroundColor White
|
|
Write-Host " Branch : $branch" -ForegroundColor White
|
|
Write-Host " Version : $Version" -ForegroundColor White
|
|
Write-Host " Label : $(if ($config.EnvLabel) { $config.EnvLabel } else { '(none)' })" -ForegroundColor White
|
|
|
|
Push-Location $repoRoot
|
|
try {
|
|
# --- 1. Build the image from the channel branch --------------------------
|
|
|
|
if (-not $SkipBuild) {
|
|
# Build from the channel branch WITHOUT switching the operator's working
|
|
# tree: create a throwaway worktree at the ref, build there, remove it.
|
|
# This keeps a release build reproducible and side-effect-free even when
|
|
# the operator has local edits checked out.
|
|
$worktree = Join-Path ([System.IO.Path]::GetTempPath()) "shopdb-build-$Version"
|
|
Write-Host "==> Fetching origin/$branch" -ForegroundColor Cyan
|
|
git fetch origin $branch
|
|
Assert-LastExit 'git fetch'
|
|
|
|
if (Test-Path $worktree) {
|
|
git worktree remove --force $worktree 2>$null
|
|
Remove-Item $worktree -Recurse -Force -ErrorAction SilentlyContinue
|
|
}
|
|
Write-Host "==> Adding build worktree at $worktree" -ForegroundColor Cyan
|
|
git worktree add --detach $worktree "origin/$branch"
|
|
Assert-LastExit 'git worktree add'
|
|
|
|
try {
|
|
Write-Host "==> Building $appImage from $branch" -ForegroundColor Cyan
|
|
docker build -t $appImage $worktree
|
|
Assert-LastExit 'docker build'
|
|
} finally {
|
|
Write-Host "==> Removing build worktree" -ForegroundColor Cyan
|
|
git worktree remove --force $worktree 2>$null
|
|
Remove-Item $worktree -Recurse -Force -ErrorAction SilentlyContinue
|
|
}
|
|
} else {
|
|
Write-Host "==> Skipping build; reusing $appImage" -ForegroundColor Yellow
|
|
}
|
|
|
|
Write-Host "==> Pulling $MysqlImage" -ForegroundColor Cyan
|
|
docker pull $MysqlImage
|
|
Assert-LastExit 'docker pull'
|
|
|
|
# --- 2. Save image bundle into the package dir ---------------------------
|
|
|
|
$pkgName = "shopdb-$Channel-$Version"
|
|
$pkgDir = Join-Path $OutDir $pkgName
|
|
if (Test-Path $pkgDir) { Remove-Item $pkgDir -Recurse -Force }
|
|
New-Item -ItemType Directory -Path $pkgDir -Force | Out-Null
|
|
|
|
$stem = "shopdb-stack-$Version"
|
|
$tar = Join-Path $pkgDir "$stem.tar"
|
|
$gz = Join-Path $pkgDir "$stem.tar.gz"
|
|
|
|
Write-Host "==> Saving $appImage + $MysqlImage" -ForegroundColor Cyan
|
|
docker save -o $tar $appImage $MysqlImage
|
|
Assert-LastExit 'docker save'
|
|
|
|
Write-Host "==> Compressing to $gz" -ForegroundColor Cyan
|
|
$inStream = [System.IO.File]::OpenRead($tar)
|
|
$outStream = [System.IO.File]::Create($gz)
|
|
$gzip = New-Object System.IO.Compression.GzipStream($outStream, [System.IO.Compression.CompressionMode]::Compress)
|
|
try {
|
|
$inStream.CopyTo($gzip)
|
|
} finally {
|
|
$gzip.Dispose(); $outStream.Dispose(); $inStream.Dispose()
|
|
}
|
|
Remove-Item $tar
|
|
|
|
$hash = (Get-FileHash $gz -Algorithm SHA256).Hash.ToLower()
|
|
"$hash $stem.tar.gz" | Set-Content -Path "$gz.sha256" -NoNewline
|
|
|
|
# --- 3. Copy the run templates into the package -------------------------
|
|
|
|
Copy-Item (Join-Path $repoRoot 'docker-compose.airgap.yml') $pkgDir
|
|
Copy-Item (Join-Path $repoRoot 'docker-compose.channel.yml') $pkgDir
|
|
|
|
# --- 4. Channel-filled .env ---------------------------------------------
|
|
# Secrets ship as explicit CHANGE_ME_* sentinels, NOT generated values. The
|
|
# site-side installer refuses to start until every sentinel is replaced, so a
|
|
# distributable can never silently go live on shipped credentials. Ports,
|
|
# image tag, and the channel label are real because they are not secrets.
|
|
|
|
$apiPort = if ($Channel -eq 'edge') { 5011 } else { 5001 }
|
|
$mysqlPort = if ($Channel -eq 'edge') { 3316 } else { 3306 }
|
|
|
|
$envLines = @(
|
|
"# ShopDB $Channel installer .env (built $(Get-Date -Format s)).",
|
|
"# REQUIRED: replace every CHANGE_ME_* value below before installing.",
|
|
"# Generate strong secrets, e.g.:",
|
|
"# python -c ""import secrets; print(secrets.token_urlsafe(64))""",
|
|
"FLASK_APP=wsgi.py",
|
|
"FLASK_ENV=production",
|
|
"SECRET_KEY=CHANGE_ME_SECRET_KEY",
|
|
"JWT_SECRET_KEY=CHANGE_ME_JWT_SECRET_KEY",
|
|
"DATABASE_URL=mysql+pymysql://shopdb:CHANGE_ME_DB_PASSWORD@db:3306/shopdb_flask?charset=utf8mb4",
|
|
"CORS_ORIGINS=https://localhost:$apiPort",
|
|
"LOG_LEVEL=INFO",
|
|
"MYSQL_ROOT_PASSWORD=CHANGE_ME_ROOT_PASSWORD",
|
|
"MYSQL_PASSWORD=CHANGE_ME_DB_PASSWORD",
|
|
"MYSQL_PORT=$mysqlPort",
|
|
"API_PORT=$apiPort",
|
|
"IMAGE_TAG=$Version",
|
|
"SHOPDB_ENV_LABEL=$($config.EnvLabel)"
|
|
)
|
|
Set-Content -Path (Join-Path $pkgDir '.env') -Value $envLines -Encoding ASCII
|
|
|
|
# --- 5. Site-side installer script --------------------------------------
|
|
# One command at the site: verify checksum, load the image, bring the stack
|
|
# up with the airgap template + channel overlay. Mirrors the run steps in
|
|
# docs/DEPLOY-AIRGAP.md but wraps them so an operator runs a single file.
|
|
|
|
$installer = @'
|
|
<#
|
|
.SYNOPSIS
|
|
Install / start this ShopDB stack on the target host (one command).
|
|
.DESCRIPTION
|
|
Verifies the image-bundle checksum, docker-loads it, and brings the stack up
|
|
from the bundled compose templates and .env. Re-runnable: on later runs it
|
|
skips the load if the image is already present and just applies the compose.
|
|
#>
|
|
[CmdletBinding()]
|
|
param([switch]$Recreate)
|
|
|
|
$ErrorActionPreference = 'Stop'
|
|
Set-Location $PSScriptRoot
|
|
|
|
if (-not (Get-Command docker -ErrorAction SilentlyContinue)) {
|
|
throw 'docker not found. Install Docker Desktop / Engine and retry.'
|
|
}
|
|
|
|
$bundle = Get-ChildItem -Filter 'shopdb-stack-*.tar.gz' | Select-Object -First 1
|
|
if (-not $bundle) { throw 'No shopdb-stack-*.tar.gz found next to this script.' }
|
|
|
|
# Refuse to start on shipped placeholder secrets. A distributable must never go
|
|
# live on credentials that ship in the box; the operator has to rotate every
|
|
# CHANGE_ME_* in .env first. This is a hard gate, not a warning.
|
|
$sentinels = Select-String -Path '.env' -Pattern 'CHANGE_ME_' -SimpleMatch
|
|
if ($sentinels) {
|
|
Write-Host 'Refusing to start: .env still has placeholder secrets.' -ForegroundColor Red
|
|
Write-Host 'Replace every CHANGE_ME_* value with a real secret, then re-run:'
|
|
foreach ($line in $sentinels) { Write-Host " $($line.Line)" -ForegroundColor Yellow }
|
|
Write-Host ''
|
|
Write-Host 'Generate strong values, e.g.:'
|
|
Write-Host ' python -c "import secrets; print(secrets.token_urlsafe(64))"'
|
|
throw 'Placeholder secrets present in .env.'
|
|
}
|
|
|
|
$sumFile = "$($bundle.Name).sha256"
|
|
if (Test-Path $sumFile) {
|
|
Write-Host "==> Verifying $($bundle.Name) checksum" -ForegroundColor Cyan
|
|
$expected = ((Get-Content $sumFile -Raw).Trim() -split '\s+')[0].ToLower()
|
|
$actual = (Get-FileHash $bundle.FullName -Algorithm SHA256).Hash.ToLower()
|
|
if ($expected -ne $actual) { throw "Checksum mismatch. Expected $expected, got $actual." }
|
|
Write-Host " OK" -ForegroundColor Green
|
|
} else {
|
|
Write-Host "==> No checksum file; skipping verification" -ForegroundColor Yellow
|
|
}
|
|
|
|
Write-Host "==> Loading image bundle" -ForegroundColor Cyan
|
|
docker load -i $bundle.FullName
|
|
if ($LASTEXITCODE -ne 0) { throw "docker load failed ($LASTEXITCODE)." }
|
|
|
|
# -p is NOT optional here. Without it compose names the project after the
|
|
# containing folder, which is shopdb-<channel>-<version> - so installing the
|
|
# next version creates a NEW project with new empty named volumes, and the site
|
|
# comes up looking wiped. The old data is still there under the old project's
|
|
# volumes, but nothing carries it across. Pin the project to the channel so an
|
|
# upgrade lands on the same volumes.
|
|
$composeArgs = @(
|
|
'compose', '--env-file', '.env',
|
|
'-p', 'shopdb-__CHANNEL__',
|
|
'-f', 'docker-compose.airgap.yml',
|
|
'-f', 'docker-compose.channel.yml'
|
|
)
|
|
if ($Recreate) { $composeArgs += @('up', '-d', '--force-recreate') }
|
|
else { $composeArgs += @('up', '-d') }
|
|
|
|
Write-Host "==> Starting the stack" -ForegroundColor Cyan
|
|
& docker @composeArgs
|
|
if ($LASTEXITCODE -ne 0) { throw "docker compose up failed ($LASTEXITCODE)." }
|
|
|
|
$apiPort = (Select-String -Path '.env' -Pattern '^API_PORT=(.+)$').Matches.Groups[1].Value
|
|
Write-Host ''
|
|
Write-Host "ShopDB is up: https://localhost:$apiPort" -ForegroundColor Green
|
|
Write-Host "Seed the first admin (once):"
|
|
Write-Host " docker compose --env-file .env -f docker-compose.airgap.yml exec api flask seed admin <user> <email> <password>"
|
|
'@
|
|
# The here-string above is single-quoted so nothing in it interpolates - that
|
|
# is deliberate, the installer is full of $ that must survive verbatim. The
|
|
# compose project name is the one value that has to be baked per channel, so
|
|
# substitute it here rather than switching the whole block to interpolation.
|
|
$installer = $installer.Replace('__CHANNEL__', $Channel)
|
|
Set-Content -Path (Join-Path $pkgDir 'Install-ShopDB.ps1') -Value $installer -Encoding ASCII
|
|
|
|
# --- 6. README ----------------------------------------------------------
|
|
|
|
$badgeLine = if ($config.EnvLabel) {
|
|
"This is the EDGE (bleeding-edge) build from '$branch'. The sidebar shows a red '$($config.EnvLabel)' badge."
|
|
} else {
|
|
"This is the STABLE build from '$branch'. No DEV badge."
|
|
}
|
|
$readme = @(
|
|
"ShopDB $Channel installer ($Version)",
|
|
"==================================================",
|
|
"",
|
|
$badgeLine,
|
|
"",
|
|
"Contents:",
|
|
" shopdb-stack-$Version.tar.gz Docker image bundle (app + MySQL)",
|
|
" shopdb-stack-$Version.tar.gz.sha256 Checksum for the bundle",
|
|
" docker-compose.airgap.yml Image-only run template",
|
|
" docker-compose.channel.yml Channel label overlay",
|
|
" .env Pre-filled config (ROTATE SECRETS)",
|
|
" Install-ShopDB.ps1 One-command installer",
|
|
"",
|
|
"Install (Windows, from this folder):",
|
|
" 1. Edit .env: replace every CHANGE_ME_* value with a real secret.",
|
|
" 2. powershell -ExecutionPolicy Bypass -File .\Install-ShopDB.ps1",
|
|
"",
|
|
"The installer REFUSES to start while any CHANGE_ME_* placeholder remains,",
|
|
"so the stack can never come up on shipped credentials.",
|
|
"",
|
|
"The API listens on https://localhost:$apiPort (self-signed cert on first",
|
|
"run). Adjust the port or CORS origins in .env if needed. Both channels can",
|
|
"run on one host at once because stable and edge use different ports and",
|
|
"compose project state.",
|
|
"",
|
|
"Generate strong secrets with:",
|
|
" python -c ""import secrets; print(secrets.token_urlsafe(64))"""
|
|
)
|
|
Set-Content -Path (Join-Path $pkgDir 'README.txt') -Value $readme -Encoding ASCII
|
|
|
|
# --- 7. Summary ---------------------------------------------------------
|
|
|
|
$sizeMb = [math]::Round((Get-Item $gz).Length / 1MB, 1)
|
|
Write-Host ''
|
|
Write-Host "Installer package ready:" -ForegroundColor Green
|
|
Write-Host " $pkgDir"
|
|
Write-Host " bundle : $stem.tar.gz ($sizeMb MB)"
|
|
Write-Host " sha256 : $hash"
|
|
Write-Host " label : $(if ($config.EnvLabel) { $config.EnvLabel } else { '(none, stable)' })"
|
|
Write-Host ''
|
|
Write-Host "Ship the folder; the operator runs Install-ShopDB.ps1."
|
|
}
|
|
finally {
|
|
Pop-Location
|
|
}
|