ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Make rollback, first upgrades and pin changes work as tested on Windows

Browse Source 
Found in an end-to-end test on a Windows 11 VM against a self-signed server.

- Rollback: a 1.20 agent re-enrolls within a second of an un-enroll, so an
  older agent pushed afterwards arrived on an enrolled PC and was refused.
  Revoke now takes {"blockreenroll": true, "reason": ...}; a blocked PC is
  never issued a credential and keeps working on the shared key until
  POST .../assets/<assetid>/allow-reenroll. The Enrollment page offers the
  block when un-enrolling and shows blocked PCs with an allow button.
- First upgrade of old agents: agents before 1.20 check the server
  certificate when downloading the upgrade, so a store wave failed on PCs that
  do not trust a self-signed server while the Migration tab said ready. A
  failed download now marks the PC blocked with "server certificate not
  trusted by this PC". New Install-ShopDBServerCertificate.ps1 (GE-Enforce,
  SYSTEM) adds a non-CA server certificate to the trusted roots after checking
  its thumbprint; the server certificate page offers a .cer download and the
  SHA-1 thumbprint.
- The generated installer writes ServerPins (with an optional next pin for a
  planned rotation), can leave the shared key out so the PC waits for
  approval, never touches an existing credential, and skips the trusted-root
  import for a CA certificate.
- The agent re-reads server pins about every 60 seconds and recycles pooled
  connections within 2 minutes, so a pin change needs no restart.
- An alert can only be acknowledged by a PC it targets.
- A wave no longer counts a PC as failed after a later wave upgraded it.
- The scripts report a pin mismatch only when the pin really is missing, and
  otherwise log the real error.
- Runbook: trust step before the first wave, rollback order, pin re-read,
  verifying the pin against the live URL; agent contract updated.
This commit is contained in:
cproudlock
2026-09-15 13:37:37 -04:00
parent 31ed9a5fde
commit da4b224ac1
20 changed files with 1713 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files

290
plugins/endpoints/api/siteagent.py
View File

@@ -21,11 +21,18 @@ The generated install.ps1 writes those two registry values from the operator's
inputs; appsettings.json ships with an empty baseurl and a placeholder apikey so
the agent fails closed until provisioned. The collector key is embedded only in
the install script's default parameter and is never stored or logged here.
Per-PC credentials: the shared key is optional per build. A package built
without it carries no key at all; the agent then asks to enroll and an admin
approves it. Every package pins the server's public key (ServerPins, the
lowercase hex SHA-256 of the leaf SubjectPublicKeyInfo), optionally with a
second "next" pin for a planned certificate rotation.
"""
import io
import json
import os
import re
import ssl
import zipfile
@@ -34,9 +41,12 @@ from flask import request, current_app, send_file
from shopdb.api import (success_response, error_response, ErrorCodes,
require_permission)
from .routes import endpoints_bp
from .routes import endpoints_bp, _spki_sha256
from .agentauth import shared_key_allowed
from . import agentstore
_PIN_PATTERN = re.compile(r'^[0-9a-f]{64}$')
# Prebuilt, staged agent binaries. Built out of band and dropped under
# instance/siteagent/ (mirrors the adoption tool's STAGED_EXE_RELPATH model).
STAGED_AGENT_RELPATH = os.path.join('siteagent', 'shopdbagent.exe')
@@ -110,6 +120,49 @@ def _validate_pem_cert(raw):
return None
return ''.join(blocks)
def _leaf_certificate_facts(pembytes):
"""(spkisha256, isca) for the FIRST certificate in a PEM, else (None, None).
The first block is the leaf in a leaf-plus-chain file, which is the cert
the agent pins and the one Import-Certificate brings into the store.
"""
from cryptography import x509
try:
text = pembytes.decode('ascii')
except (UnicodeDecodeError, AttributeError):
return None, None
start = text.find('-----BEGIN CERTIFICATE-----')
end = text.find('-----END CERTIFICATE-----', start)
if start == -1 or end == -1:
return None, None
block = (text[start:end + len('-----END CERTIFICATE-----')] + '\n').encode('ascii')
try:
certificate = x509.load_pem_x509_certificate(block)
except ValueError:
return None, None
try:
isca = bool(certificate.extensions.get_extension_for_class(
x509.BasicConstraints).value.ca)
except x509.ExtensionNotFound:
isca = False
return _spki_sha256(block), isca
def _current_server_pin():
"""Pin of this server's own leaf certificate, or None when not found/CA."""
path = _server_cert_path()
if not path:
return None
try:
with open(path, 'rb') as handle:
pin, isca = _leaf_certificate_facts(handle.read())
except OSError:
return None
return None if isca else pin
# The non-secret runtime config shipped in the package. baseurl and apikey stay
# empty/placeholder so the agent fails closed until the registry is provisioned
# by install.ps1. Cadence is stamped from the operator's input at build time.
@@ -175,20 +228,42 @@ FILES
USAGE
1. Copy this whole folder to the target endpoint.
2. Right-click install.ps1 and Run with PowerShell as administrator. It
writes the server URL and collector key to HKLM\\SOFTWARE\\GE\\ShopDB,
{provision} to HKLM\\SOFTWARE\\GE\\ShopDB,
copies the agent into Program Files, and starts the service. It also
registers the tray to autostart for every user at logon and adds a
"ShopDB Agent" Start Menu shortcut for manual relaunch.
{enrollment}
UNINSTALL
Run uninstall-shopdbagent.ps1 from an elevated PowerShell to fully remove the
agent, tray autostart, Start Menu shortcut, and credentials.
SECURITY
install.ps1 carries a live collector key in its default -CollectorKey value.
Treat this package as a secret. The key is not recoverable from the ShopDB
server after it was generated; regenerate the key in the UI to revoke it.
"""
{security}"""
_README_SHAREDKEY = {
True: {
'provision': 'writes the server URL, server pins, and collector key',
'enrollment': '',
'security': (
' install.ps1 carries a live collector key in its default -CollectorKey value.\n'
' Treat this package as a secret. The key is not recoverable from the ShopDB\n'
' server after it was generated; regenerate the key in the UI to revoke it.\n'),
},
False: {
'provision': 'writes the server URL and server pins',
'enrollment': (
' This package carries NO shared key. After install the PC requests\n'
' enrollment; approve it in Settings > Endpoints > Enrollment.\n'),
'security': (
' This package holds no secret. Each PC receives its own credential when\n'
' an admin approves its enrollment request; an existing enrolled\n'
' credential on the PC is never touched by install.ps1.\n'),
},
}
def _readme(includesharedkey):
return _README.format(**_README_SHAREDKEY[bool(includesharedkey)])
def _staged_path(relpath):
@@ -196,7 +271,9 @@ def _staged_path(relpath):
return os.path.join(current_app.instance_path, relpath)
def _install_script(baseurl, collectorkey, cadenceseconds, certfilename=None):
def _install_script(baseurl, collectorkey, cadenceseconds, certfilename=None,
serverpins=None, includesharedkey=True,
trustcertinroot=True):
"""The install.ps1 that provisions credentials and installs the service.
The operator's server URL and collector key are stamped as the default
@@ -206,10 +283,34 @@ def _install_script(baseurl, collectorkey, cadenceseconds, certfilename=None):
certificate is bundled, certfilename names it and the script imports it into
the machine Trusted Root store so the agent's outbound TLS to a self-signed
or internal-CA server validates.
serverpins are written as REG_MULTI_SZ ServerPins. With includesharedkey
False the collector key is left out of the script entirely (no parameter,
no registry write) and the installer tells the operator to approve the
enrollment request. trustcertinroot False (a CA certificate) skips the Root
import; the script also re-checks basicConstraints on the endpoint.
"""
baseurl_literal = json.dumps(baseurl)
collectorkey_literal = json.dumps(collectorkey)
certfile_literal = json.dumps(certfilename or '')
pins_literal = '@(' + ', '.join(json.dumps(pin) for pin in (serverpins or [])) + ')'
if includesharedkey:
keyparam = ' [string] $CollectorKey = {0},\n'.format(json.dumps(collectorkey))
keycheck = (
'if ([string]::IsNullOrWhiteSpace($CollectorKey)) {\n'
' throw "CollectorKey is not set. Regenerate the key in the endpoints UI and rebuild."\n'
'}\n')
keywrite = 'Set-HklmValueBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "CollectorKey" -Value $CollectorKey\n'
sharedkeymode = 'included'
enrollnotice = ''
else:
keyparam = ''
keycheck = ''
keywrite = (
'Write-Host " no shared key in this package; the agent will request enrollment." -ForegroundColor Yellow\n')
sharedkeymode = 'omitted'
enrollnotice = (
'Write-Host " This PC will request enrollment. Approve it in '
'Settings > Endpoints > Enrollment." -ForegroundColor Yellow\n')
return """# install.ps1
# ShopDB endpoint agent installer. Run from an ELEVATED (admin) PowerShell on
# the endpoint. Provisions the registry credentials, copies the agent to
@@ -219,7 +320,7 @@ def _install_script(baseurl, collectorkey, cadenceseconds, certfilename=None):
[CmdletBinding()]
param(
[string] $BaseUrl = {baseurl},
[string] $CollectorKey = {collectorkey},
{keyparam} [string[]] $ServerPins = {serverpins},
[string] $InstallDir = "C:\\Program Files\\ShopDB\\EndpointAgent",
[string] $ServiceName = "ShopDbEndpointAgent",
[string] $TrayExeName = "shopdb-endpoint-tray.exe",
@@ -251,9 +352,8 @@ if (-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administra
if ([string]::IsNullOrWhiteSpace($BaseUrl)) {{
throw "BaseUrl is not set. Pass -BaseUrl 'http://<host>:<port>' when running the script."
}}
if ([string]::IsNullOrWhiteSpace($CollectorKey)) {{
throw "CollectorKey is not set. Regenerate the key in the endpoints UI and rebuild."
}}
{keycheck}$sharedKeyMode = "{sharedkeymode}"
$trustCertInRoot = {trustcert}
# Write HKLM values to BOTH the 64-bit and 32-bit (WOW6432Node) registry views.
# The agent reads Registry64 first then Registry32 and uses the first view that
@@ -280,10 +380,55 @@ function Set-HklmValueBothViews {{
}}
}}
# REG_MULTI_SZ in both views, compare-before-write so a reinstall with the same
# pins leaves the value (and its last-write time) alone.
function Set-HklmMultiStringBothViews {{
param(
[Parameter(Mandatory)] [string] $SubKey,
[Parameter(Mandatory)] [string] $Name,
[Parameter(Mandatory)] [string[]] $Value
)
$wanted = ($Value -join "`n")
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64,
[Microsoft.Win32.RegistryView]::Registry32)) {{
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey(
[Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {{
$key = $base.CreateSubKey($SubKey)
try {{
$same = $false
$existingValue = $key.GetValue($Name, $null)
if ($null -ne $existingValue -and
$key.GetValueKind($Name) -eq [Microsoft.Win32.RegistryValueKind]::MultiString) {{
$same = ((@($existingValue) -join "`n") -ceq $wanted)
}}
if ($same) {{
Write-Host " $Name unchanged ($view)"
}}
else {{
$key.SetValue($Name, [string[]] $Value, [Microsoft.Win32.RegistryValueKind]::MultiString)
Write-Host " $Name written ($view)"
}}
}}
finally {{ $key.Dispose() }}
}}
finally {{ $base.Dispose() }}
}}
}}
Write-Host "== ShopDB endpoint agent install ==" -ForegroundColor Cyan
Write-Host "BaseUrl : $BaseUrl"
Write-Host "InstallDir : $InstallDir"
Write-Host "ServiceName : $ServiceName"
Write-Host "Shared key : $sharedKeyMode"
# Per-PC credential lives under ProgramData\\ShopDB\\EndpointAgent\\credential and
# belongs to the agent. This installer never writes or deletes it, so a reinstall
# or upgrade keeps an enrolled PC enrolled.
$credentialDir = Join-Path $env:ProgramData "ShopDB\\EndpointAgent\\credential"
if (Test-Path (Join-Path $credentialDir "credential.bin")) {{
Write-Host " existing enrolled credential found; kept as is."
}}
# 0. Stop and remove any previous install so the file copy is a clean overwrite.
$existing = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
@@ -306,21 +451,48 @@ foreach ($name in @("shopdbagent", $trayBase)) {{
# 1. Provision registry credentials (both 64-bit and 32-bit views).
Write-Host "[1/6] Provisioning registry HKLM:\\SOFTWARE\\GE\\ShopDB (Registry64 + Registry32) ..."
Set-HklmValueBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "BaseUrl" -Value $BaseUrl
Set-HklmValueBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "CollectorKey" -Value $CollectorKey
{keywrite}
# Server public-key pins (current, plus next during a planned rotation). An
# empty list leaves any ServerPins already on the PC untouched.
$pinList = @($ServerPins | Where-Object {{ -not [string]::IsNullOrWhiteSpace($_) }})
if ($pinList.Count -gt 0) {{
Set-HklmMultiStringBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "ServerPins" -Value $pinList
}}
else {{
Write-Warning "no server pin in this package; ServerPins left as is."
}}
# 1a. Trust the bundled server certificate, if one shipped. The agent's outbound
# TLS to a self-signed or internal-CA server fails closed until the machine
# trusts the cert. Importing into LocalMachine\\Root covers the SYSTEM service.
# No-op when the deployment uses a publicly trusted (or already-trusted internal)
# certificate, in which case no cert is bundled.
# Kept for agents older than 1.20 and PowerShell download paths that do not pin.
# Only a leaf is trusted: a CA in Root would trust everything it signs.
if (-not [string]::IsNullOrWhiteSpace($CertFileName)) {{
$certPath = Join-Path $sourceDir $CertFileName
if (Test-Path $certPath) {{
Write-Host " trusting bundled server certificate $CertFileName ..."
try {{
Import-Certificate -FilePath $certPath -CertStoreLocation Cert:\\LocalMachine\\Root | Out-Null
$certIsCa = -not $trustCertInRoot
if (-not $certIsCa) {{
try {{
$probe = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($certPath)
foreach ($ext in $probe.Extensions) {{
if ($ext -is [System.Security.Cryptography.X509Certificates.X509BasicConstraintsExtension] -and
$ext.CertificateAuthority) {{ $certIsCa = $true }}
}}
}}
catch {{ }}
}}
if ($certIsCa) {{
Write-Warning "server certificate $CertFileName is a CA certificate; not importing it into LocalMachine\\Root."
}}
else {{
Write-Host " trusting bundled server certificate $CertFileName ..."
try {{
Import-Certificate -FilePath $certPath -CertStoreLocation Cert:\\LocalMachine\\Root | Out-Null
}}
catch {{ Write-Warning "could not import server certificate: $($_.Exception.Message)" }}
}}
catch {{ Write-Warning "could not import server certificate: $($_.Exception.Message)" }}
}}
else {{
Write-Warning "server certificate $CertFileName named but not found beside install.ps1."
@@ -578,7 +750,7 @@ if (-not $serviceOk) {{
throw "Service $ServiceName did not reach Running after 5 attempts. Leaving install.ps1 in place for diagnosis; re-run once resolved."
}}
Write-Host " service is running." -ForegroundColor Green
{enrollnotice}
$trayBase2 = [System.IO.Path]::GetFileNameWithoutExtension($TrayExeName)
if (-not $NoTray -and (Test-Path (Join-Path $InstallDir $TrayExeName))) {{
# Report only; never relaunch from here. This block used to Start-Process the
@@ -601,8 +773,8 @@ if (-not $NoTray -and (Test-Path (Join-Path $InstallDir $TrayExeName))) {{
# 5. Preserve the install artifacts (minus the secret-bearing install.ps1) under
# ProgramData so the uninstaller and appsettings/cert are available later, then
# scrub install.ps1 from wherever it was run. install.ps1 carries the live
# collector key in its default parameter, so it must not linger on the endpoint.
# scrub install.ps1 from wherever it was run. A shared-key build carries the live
# key in its default parameter, so it must not linger on the endpoint.
Write-Host "[5/6] Archiving install artifacts and removing install.ps1 ..."
$dataDir = Join-Path $env:ProgramData "ShopDB\\EndpointAgent"
New-Item -ItemType Directory -Force -Path $dataDir | Out-Null
@@ -618,8 +790,8 @@ $uninstaller = Join-Path $dataDir "uninstall-shopdbagent.ps1"
$installSelf = Join-Path $sourceDir "install.ps1"
if (Test-Path $installSelf) {{
try {{ Remove-Item -Path $installSelf -Force -ErrorAction Stop; Write-Host " removed install.ps1 (carried the collector key)." }}
catch {{ Write-Warning "could not remove install.ps1 - delete it manually, it contains the collector key: $($_.Exception.Message)" }}
try {{ Remove-Item -Path $installSelf -Force -ErrorAction Stop; Write-Host " removed install.ps1." }}
catch {{ Write-Warning "could not remove install.ps1 - delete it manually: $($_.Exception.Message)" }}
}}
Write-Host "[6/6] Done." -ForegroundColor Green
@@ -639,7 +811,7 @@ try {{
$ledgerTrayVersion = "not-installed"
$ledgerTrayPath = Join-Path $InstallDir $TrayExeName
if (Test-Path $ledgerTrayPath) {{ $ledgerTrayVersion = (Get-Item $ledgerTrayPath).VersionInfo.FileVersion }}
$ledgerLine = "{{0}} host={{1}} installer={{2}} service={{3}} tray={{4}} dir={{5}} usercontexttask={{6}}" -f (Get-Date -Format "yyyy-MM-dd HH:mm:ss zzz"), $env:COMPUTERNAME, $env:USERNAME, $ledgerAgentVersion, $ledgerTrayVersion, $InstallDir, $taskState
$ledgerLine = "{{0}} host={{1}} installer={{2}} service={{3}} tray={{4}} dir={{5}} usercontexttask={{6}} sharedkey={{7}} serverpins={{8}}" -f (Get-Date -Format "yyyy-MM-dd HH:mm:ss zzz"), $env:COMPUTERNAME, $env:USERNAME, $ledgerAgentVersion, $ledgerTrayVersion, $InstallDir, $taskState, $sharedKeyMode, $pinList.Count
Add-Content -Path $ledgerPath -Value $ledgerLine -Encoding UTF8
Write-Host "Install ledger: $ledgerPath"
Write-Host " $ledgerLine" -ForegroundColor DarkGray
@@ -655,7 +827,13 @@ else {{
}}
""".format(
baseurl=baseurl_literal,
collectorkey=collectorkey_literal,
keyparam=keyparam,
keycheck=keycheck,
keywrite=keywrite,
sharedkeymode=sharedkeymode,
enrollnotice=enrollnotice,
serverpins=pins_literal,
trustcert='$true' if trustcertinroot else '$false',
certfile=certfile_literal,
)
@@ -672,6 +850,8 @@ def _siteagent_status_payload():
'agentstaged': agent_staged,
'traystaged': tray_staged,
'certavailable': _server_cert_path() is not None,
'serverpin': _current_server_pin(),
'sharedkeyallowed': shared_key_allowed(),
'agentcurrent': agentstore.current_version('agent'),
'traycurrent': agentstore.current_version('tray'),
'agentversions': agentstore.list_versions('agent'),
@@ -694,7 +874,12 @@ def package_siteagent():
"""Bundle the site agent as a downloadable ZIP.
Accepts either JSON or multipart/form-data with fields baseurl,
collectorkey, cadenceseconds, and bundlecert. A multipart request may also
collectorkey, cadenceseconds, bundlecert, includesharedkey, and
nextserverpin. includesharedkey defaults to the site setting
plugin.endpoints.sharedkeyallowed; when off, collectorkey is ignored and
nothing in the package carries it. nextserverpin (64 lowercase hex) is
shipped beside the current server pin for a planned certificate rotation.
A multipart request may also
carry a certfile upload: an operator-supplied .crt/.pem that is validated as
X.509 and bundled so the installer trusts it as a root on the endpoint. This
is the reverse-proxy path where the app cannot see its own TLS cert. When no
@@ -717,11 +902,25 @@ def package_siteagent():
collectorkey = (source.get('collectorkey') or '').strip()
cadenceseconds = source.get('cadenceseconds', 900)
bundlecert = str(source.get('bundlecert', '')).lower() in ('1', 'true', 'yes', 'on')
rawinclude = source.get('includesharedkey')
if rawinclude is None or str(rawinclude).strip() == '':
includesharedkey = shared_key_allowed()
else:
includesharedkey = str(rawinclude).lower() in ('1', 'true', 'yes', 'on')
nextserverpin = (source.get('nextserverpin') or '').strip()
if not baseurl or not collectorkey:
if not includesharedkey:
collectorkey = ''
if not baseurl or (includesharedkey and not collectorkey):
return error_response(
ErrorCodes.VALIDATION_ERROR,
'baseurl and collectorkey are required',
'baseurl and collectorkey are required' if includesharedkey
else 'baseurl is required',
http_code=400)
if nextserverpin and not _PIN_PATTERN.match(nextserverpin):
return error_response(
ErrorCodes.VALIDATION_ERROR,
'nextserverpin must be 64 lowercase hex characters',
http_code=400)
try:
@@ -739,6 +938,8 @@ def package_siteagent():
appsettings = dict(_BASE_APPSETTINGS)
appsettings['cadenceseconds'] = cadenceseconds
if not includesharedkey:
appsettings['apikey'] = ''
# An operator-uploaded cert wins over the auto-detected server cert: it is
# the explicit reverse-proxy answer where the app cannot see its own TLS
@@ -758,14 +959,43 @@ def package_siteagent():
certfilename = 'server.crt' if (cert_pem or certpath) else None
# Pin the bundled leaf when there is one; a CA bundle cannot be pinned (the
# agent hashes the leaf) so fall back to this server's own certificate.
trustcertinroot = True
currentpin = None
if certfilename:
if cert_pem:
bundledbytes = cert_pem.encode('ascii')
else:
with open(certpath, 'rb') as handle:
bundledbytes = handle.read()
bundledpin, bundledisca = _leaf_certificate_facts(bundledbytes)
trustcertinroot = not bundledisca
if not bundledisca:
currentpin = bundledpin
if currentpin is None:
currentpin = _current_server_pin()
if nextserverpin and currentpin is None:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'nextserverpin needs a current server pin; bundle or upload the '
'server leaf certificate',
http_code=400)
serverpins = [pin for pin in (currentpin, nextserverpin) if pin]
serverpins = list(dict.fromkeys(serverpins))
appsettings['serverpins'] = serverpins
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, 'w', zipfile.ZIP_DEFLATED) as archive:
archive.writestr('appsettings.json',
json.dumps(appsettings, indent=2) + '\n')
archive.writestr('install.ps1',
_install_script(baseurl.rstrip('/'), collectorkey,
cadenceseconds, certfilename))
archive.writestr('README.txt', _README)
cadenceseconds, certfilename,
serverpins=serverpins,
includesharedkey=includesharedkey,
trustcertinroot=trustcertinroot))
archive.writestr('README.txt', _readme(includesharedkey))
if os.path.isfile(UNINSTALLER_PATH):
archive.write(UNINSTALLER_PATH, 'uninstall-shopdbagent.ps1')