Files
shopdb-flask/plugins/endpoints/api/bootstrap.py
cproudlock 066a19052b
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / gates (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / lean-build (push) Has been cancelled
Act on the wj architectural review: races, limits, pool and contract gaps
Findings F2, F4, F5, F6, F7, F8, F9, F10, F11 and F13 from
wj-architectural-review.md. F1/F1a (the non-sargable LOWER pattern) and F3
(the warranty sync holding a pooled connection) are being handled separately
and are untouched here.

- A collector check-in that raced another for the same PC lost everything:
  the unique index on the application fingerprint raised, nothing caught it,
  and the whole check-in rolled back, discarding state, events and hardware,
  answering the agent with form copy meant for a person. The insert now runs
  in a savepoint, reconciles what the other transaction already wrote, one row
  per savepoint so a second collision skips a row rather than the batch, and
  the check-in commits with a warning. This is the shape the delete side of
  the same function already had.
- Enrollment is deliberately unauthenticated, so it now carries a per-address
  rate limit and a cap on pending requests, and the cap is a setting an admin
  can see and change. A flood can no longer bury the approval queue or grow
  the table without bound.
- The fleet alert poll no longer formats debug strings on every request.
- BusinessUnit is on the contract surface (0.34.0), so the endpoints plugin
  stops reaching into a core table with raw SQL.
- A test walks every asset type descriptor and proves the endpoint it names
  really refuses a caller without the declared permission.
- The plugin settings routes now need a permission; they were telling anyone
  which modules a site runs and where its components live.
- The connection pool is sized explicitly with a short timeout, and the
  arithmetic against the server's connection ceiling is written next to the
  worker count instead of being left to defaults nobody chose.
- Fleet alerts carry both the correct lowercase keys and the old camelCase
  pair. Trays older than 1.7.0 match keys case-sensitively and would have gone
  quiet on every PC not yet upgraded; the old pair goes when they are.
- The dev server proxy points at localhost again, with environment variables
  for the remote-box workflow, and one developer's address is out of the
  server certificate page.
2026-09-16 08:57:04 -04:00

560 lines
22 KiB
Python

"""One-line bootstrap installer for the endpoint agent.
An admin mints a short-lived token in Settings > Endpoints and hands a tech one
command. On a PC with no agent, in an ELEVATED PowerShell: the command
downloads bootstrap.ps1 to a file, runs it, and deletes it again.
That installs the agent and tray with NO shared key, so the PC asks to enroll
and waits for an admin to approve it in Settings > Endpoints > Enrollment. This
is the path for a ONE-OFF PC; a managed bay keeps getting the agent from
GE-Enforce, and the generated script says so when it finds an agent already
installed.
Routes
POST /siteagent/bootstrap/tokens admin, audited; secret once
GET /siteagent/bootstrap/tokens admin; never a secret
POST /siteagent/bootstrap/tokens/<id>/revoke admin, audited
GET /siteagent/bootstrap.ps1?t=... token only; not a use
GET /siteagent/bootstrap/package?t=... token only; one use
The two token-authenticated routes answer 404 for anything not currently
usable - unknown, expired, revoked or exhausted - so the URL confirms nothing
about which tokens exist. Both are per-IP rate limited with the same fixed
window shape the login and payload-download limiters use, and both write an
audit row carrying the caller's IP.
"""
import hashlib
import json
import re
import secrets
from datetime import datetime, timedelta, timezone
from flask import Response, current_app, request, send_file
from shopdb.api import (
db, AuditLog, Setting, client_ip, error_response, success_response,
ErrorCodes, require_role,
)
from ..models import EndpointBootstrapToken
from . import siteagent
from .ratelimit import ratelimited, rate_limit_response
from .routes import endpoints_bp
# sdbb_ + 43 chars base64url (32 random bytes, unpadded). Same shape as the
# per-PC credential, different prefix so a leaked value is identifiable.
TOKEN_PREFIX = 'sdbb_'
_TOKEN_PATTERN = re.compile(r'^sdbb_[A-Za-z0-9_-]{43}$')
DISPLAY_PREFIX_LENGTH = 12
DEFAULT_HOURS = 24
DEFAULT_MAX_USES = 25
MINIMUM_HOURS = 1
MAXIMUM_HOURS = 168
MINIMUM_USES = 1
MAXIMUM_USES = 500
LIST_LIMIT = 100
SCRIPT_PATH = '/api/endpoints/siteagent/bootstrap.ps1'
PACKAGE_PATH = '/api/endpoints/siteagent/bootstrap/package'
# Where the one-line command drops the script before running it. A literal, not
# $env:TEMP: the line has to survive being pasted into an already-running
# PowerShell, which expands every $ inside the outer double quotes before the
# child process sees them. The install needs elevation anyway, so this path is
# writable by definition.
COMMAND_SCRIPT_PATH = r'C:\Windows\Temp\shopdb-bootstrap.ps1'
# Fixed-window per-IP limiter, same shape and cache extension as the login
# limiter in shopdb.core.api.auth and the GE-Enforce payload limiter. A tech
# fetching a script and a package is two hits; a scanner walking token guesses
# is thousands.
BOOTSTRAP_RATELIMIT_MAX = 30
BOOTSTRAP_RATELIMIT_WINDOW_SECONDS = 60
# The endpoint-side force switch. Set it in the same command to reinstall over
# an agent that is already there.
FORCE_VARIABLE = 'SHOPDB_BOOTSTRAP_FORCE'
_HOST_PATTERN = re.compile(r'^[A-Za-z0-9.-]+(?::[0-9]{1,5})?$')
def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def _caller_ip():
return client_ip(request) or 'unknown'
def _ratelimited():
"""True when this caller is over budget for the current window.
The implementation moved to .ratelimit so the enrollment route shares it
rather than carrying a second copy. Budget name, config keys and defaults
are unchanged, so the behaviour here is identical.
"""
return ratelimited('endpointbootstrapratelimit',
'ENDPOINTS_BOOTSTRAP_RATELIMIT',
BOOTSTRAP_RATELIMIT_MAX,
BOOTSTRAP_RATELIMIT_WINDOW_SECONDS)
def _rate_limit_response():
return rate_limit_response('Too many bootstrap requests. Try again later.')
def _not_found():
"""The single answer for every unusable token, so nothing is confirmed."""
return error_response(ErrorCodes.NOT_FOUND, 'not found', http_code=404)
def _safe_host(value):
host = (value or '').strip()
return host if host and _HOST_PATTERN.match(host) else None
def bootstrap_base_url():
"""Base URL baked into the generated command and script.
Prefers the configured site_base_url, which already carries the scheme and
the mount path. Falls back to the forwarded host (only when a proxy is
actually declared) or the request Host plus the script root, forced to
https - the same resolution the printer installer batch generator uses,
and for the same reason: behind a reverse proxy the app sees http on a
loopback port and its url_root drops the mount.
"""
base = (Setting.get('site_base_url') or '').strip().rstrip('/')
if base:
return base
host = None
if current_app.config.get('TRUSTED_PROXY_COUNT'):
host = _safe_host(request.headers.get('X-Forwarded-Host'))
if not host:
host = _safe_host(request.host)
root = (request.script_root or '').rstrip('/')
if not host:
return root
return 'https://{0}{1}'.format(host, root)
def bootstrap_script_url(secret, baseurl=None):
"""The token URL that serves the script: downloaded by the command, and
offered in the UI as a plain file a tech can save and run instead."""
base = (baseurl if baseurl is not None else bootstrap_base_url()).rstrip('/')
return '{0}{1}?t={2}'.format(base, SCRIPT_PATH, secret)
def bootstrap_command(secret, baseurl=None):
"""The one line a tech pastes into an elevated PowerShell.
Download to a file and run it, never `irm ... | iex`: Microsoft Defender
scores the piped-to-iex shape as Trojan:Win32/Commando.A!ml and kills the
process before a line of it runs. Every bay runs Defender.
Single quotes inside and no $ anywhere, so the one line means the same
whether it is pasted into cmd.exe or into a PowerShell prompt. The download
and the run are wrapped in try/finally so the dropped file goes away even
when the script exits non-zero or the download itself fails.
"""
return ('powershell -ExecutionPolicy Bypass -c "'
"try {{ Invoke-WebRequest -UseBasicParsing '{0}' -OutFile '{1}'; "
"& '{1}' }} finally {{ Remove-Item '{1}' -Force "
'-ErrorAction SilentlyContinue }}"'
).format(bootstrap_script_url(secret, baseurl),
COMMAND_SCRIPT_PATH)
def _hash_token(secret):
return hashlib.sha256(secret.encode('utf-8')).hexdigest()
def _resolve_token():
"""The usable token named by ?t=, else None.
Unknown, malformed, expired, revoked and exhausted all resolve to None so
the caller can answer 404 for every one of them.
"""
secret = (request.args.get('t') or '').strip()
if not _TOKEN_PATTERN.match(secret):
return None
row = EndpointBootstrapToken.query.filter_by(
tokenhash=_hash_token(secret)).first()
if row is None or not row.isusable(_naive_utcnow()):
return None
return row
def _audit(action, row, details):
AuditLog.log(action, 'EndpointBootstrapToken', entityid=row.tokenid,
entityname=row.tokenprefix,
details='{0} from {1}'.format(details, _caller_ip()))
# ---------------------------------------------------------------------------
# The generated script
# ---------------------------------------------------------------------------
# Why the wait loop watches a LOCAL file rather than a server route: there is no
# public "am I approved yet" endpoint, and adding one would hand an anonymous
# caller a way to probe enrollment state. The agent writes credential.bin the
# moment it collects an approved credential, so the file appearing IS the
# enrollment, observed without asking the server anything.
_SCRIPT_TEMPLATE = '''# ShopDB endpoint agent bootstrap
#
# Generated by the ShopDB server for one short-lived install token. Run from an
# ELEVATED Windows PowerShell. It downloads this server's agent package, checks
# the binaries against the hashes stamped in below, installs the agent and tray
# with NO shared key, and leaves the PC waiting for an admin to approve its
# enrollment in Settings > Endpoints > Enrollment.
#
# A managed bay should get the agent from GE-Enforce instead. This is the
# one-off path.
$ErrorActionPreference = "Stop"
$BootstrapPackageUrl = {packageurl}
$BootstrapServiceName = "ShopDbEndpointAgent"
$BootstrapInstallDir = "C:\\Program Files\\ShopDB\\EndpointAgent"
$BootstrapAgentSha256 = {agentsha}
$BootstrapTraySha256 = {traysha}
$BootstrapForceVariable = "{forcevariable}"
$BootstrapWaitSeconds = {waitseconds}
$BootstrapCredentialPath = Join-Path $env:ProgramData "ShopDB\\EndpointAgent\\credential\\credential.bin"
Write-Host "== ShopDB endpoint agent bootstrap ==" -ForegroundColor Cyan
# 1. Elevation. Every later step writes HKLM and Program Files.
$bootstrapIdentity = [Security.Principal.WindowsIdentity]::GetCurrent()
$bootstrapPrincipal = New-Object Security.Principal.WindowsPrincipal($bootstrapIdentity)
if (-not $bootstrapPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) {{
Write-Host "This installer must run elevated." -ForegroundColor Red
Write-Host "Open PowerShell as administrator (right-click Windows PowerShell, Run as administrator) and paste the command again."
exit 1
}}
# 2. An agent already here means this PC is probably managed.
$bootstrapForce = $false
$bootstrapForceValue = [Environment]::GetEnvironmentVariable($BootstrapForceVariable)
if ($bootstrapForceValue -and $bootstrapForceValue -ne "0") {{ $bootstrapForce = $true }}
$bootstrapExisting = Get-Service -Name $BootstrapServiceName -ErrorAction SilentlyContinue
if ($bootstrapExisting -and -not $bootstrapForce) {{
Write-Host ("The ShopDB endpoint agent is already installed here (service " + $BootstrapServiceName + ", " + $bootstrapExisting.Status + ").") -ForegroundColor Yellow
Write-Host "A managed bay should get the agent from GE-Enforce, not from this one-line installer."
Write-Host ("To reinstall anyway, run this first in the same window, then paste the command again:")
Write-Host (" " + '$env:' + $BootstrapForceVariable + " = '1'")
exit 2
}}
$bootstrapAlreadyEnrolled = Test-Path $BootstrapCredentialPath
# 3. Download the package over TLS with normal chain validation. Nothing here
# relaxes certificate checking: a server this PC does not trust must be fixed,
# not bypassed.
try {{
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
}}
catch {{ }}
$bootstrapStamp = [DateTime]::UtcNow.ToString("yyyyMMddHHmmssfff")
$bootstrapTempDir = Join-Path $env:TEMP ("shopdb-bootstrap-" + $bootstrapStamp)
$bootstrapZipPath = Join-Path $bootstrapTempDir "package.zip"
$bootstrapExtractDir = Join-Path $bootstrapTempDir "package"
function Remove-BootstrapTemp {{
if (Test-Path $bootstrapTempDir) {{
try {{ Remove-Item -Path $bootstrapTempDir -Recurse -Force -ErrorAction Stop }}
catch {{ Write-Warning ("could not remove " + $bootstrapTempDir + "; delete it manually.") }}
}}
}}
function Test-BootstrapHash {{
param(
[Parameter(Mandatory)] [string] $Path,
[AllowEmptyString()] [string] $Expected,
[Parameter(Mandatory)] [string] $Label
)
$present = Test-Path $Path
if ([string]::IsNullOrWhiteSpace($Expected)) {{
# The server staged no such binary, so the package must not carry one.
if ($present) {{
Write-Host ($Label + " is in the package but this server staged none.") -ForegroundColor Red
return $false
}}
Write-Host (" " + $Label + ": not shipped by this server")
return $true
}}
if (-not $present) {{
Write-Host ($Label + " is missing from the downloaded package.") -ForegroundColor Red
return $false
}}
$actual = (Get-FileHash -Path $Path -Algorithm SHA256).Hash.ToLowerInvariant()
if ($actual -ne $Expected.ToLowerInvariant()) {{
Write-Host ($Label + " does not match the expected SHA256.") -ForegroundColor Red
Write-Host (" expected " + $Expected)
Write-Host (" actual " + $actual)
return $false
}}
Write-Host (" " + $Label + ": sha256 verified")
return $true
}}
try {{
New-Item -ItemType Directory -Force -Path $bootstrapTempDir | Out-Null
Write-Host "[1/4] Downloading the agent package ..."
Invoke-WebRequest -Uri $BootstrapPackageUrl -OutFile $bootstrapZipPath -UseBasicParsing
Write-Host "[2/4] Expanding and verifying ..."
Expand-Archive -Path $bootstrapZipPath -DestinationPath $bootstrapExtractDir -Force
$bootstrapOk = $true
if (-not (Test-BootstrapHash -Path (Join-Path $bootstrapExtractDir "shopdbagent.exe") -Expected $BootstrapAgentSha256 -Label "shopdbagent.exe")) {{ $bootstrapOk = $false }}
if (-not (Test-BootstrapHash -Path (Join-Path $bootstrapExtractDir "shopdb-endpoint-tray.exe") -Expected $BootstrapTraySha256 -Label "shopdb-endpoint-tray.exe")) {{ $bootstrapOk = $false }}
if (-not $bootstrapOk) {{
Remove-BootstrapTemp
Write-Host "Refusing to install an unverified package." -ForegroundColor Red
exit 3
}}
$bootstrapInstaller = Join-Path $bootstrapExtractDir "install.ps1"
if (-not (Test-Path $bootstrapInstaller)) {{
Remove-BootstrapTemp
Write-Host "The downloaded package carries no install.ps1." -ForegroundColor Red
exit 3
}}
# No key argument: this package has no -CollectorKey parameter at all.
Write-Host "[3/4] Installing ..."
& powershell.exe -NoProfile -ExecutionPolicy Bypass -File $bootstrapInstaller
$bootstrapInstallExit = $LASTEXITCODE
}}
catch {{
Remove-BootstrapTemp
Write-Host ("Bootstrap failed: " + $_.Exception.Message) -ForegroundColor Red
exit 3
}}
Remove-BootstrapTemp
# 4. Report what actually landed. The installer's own exit code is not enough:
# a PowerShell -File run can return 0 after a terminating error, so the service
# is read back directly.
Write-Host "[4/4] Result"
$bootstrapAgentPath = Join-Path $BootstrapInstallDir "shopdbagent.exe"
$bootstrapVersion = "not installed"
if (Test-Path $bootstrapAgentPath) {{
$bootstrapVersion = (Get-Item $bootstrapAgentPath).VersionInfo.FileVersion
}}
$bootstrapService = Get-Service -Name $BootstrapServiceName -ErrorAction SilentlyContinue
$bootstrapState = "not installed"
if ($bootstrapService) {{ $bootstrapState = [string] $bootstrapService.Status }}
Write-Host (" installed version : " + $bootstrapVersion)
Write-Host (" service state : " + $bootstrapState)
if (-not $bootstrapService -or $bootstrapState -ne "Running") {{
Write-Host "The agent service is not running. The install did not complete." -ForegroundColor Red
if ($bootstrapInstallExit) {{ Write-Host (" installer exit code: " + $bootstrapInstallExit) }}
exit 4
}}
if ($bootstrapAlreadyEnrolled) {{
Write-Host "This PC already holds an endpoint credential; no approval is needed." -ForegroundColor Green
exit 0
}}
Write-Host "This PC is waiting for approval - an admin approves it in Settings > Endpoints > Enrollment." -ForegroundColor Yellow
# Watch for the credential the agent writes once an admin approves it. Purely
# local: nothing is asked of the server, and a PC still pending after the wait
# is a normal outcome, not a failure.
$bootstrapDeadline = (Get-Date).AddSeconds($BootstrapWaitSeconds)
$bootstrapEnrolled = $false
while ((Get-Date) -lt $bootstrapDeadline) {{
if (Test-Path $BootstrapCredentialPath) {{ $bootstrapEnrolled = $true; break }}
Start-Sleep -Seconds 5
}}
if ($bootstrapEnrolled) {{
Write-Host "approved and enrolled" -ForegroundColor Green
}}
else {{
Write-Host "Still pending. The agent keeps asking on its own; nothing else is needed on this PC."
}}
exit 0
'''
def build_bootstrap_script(secret, baseurl=None):
"""The PowerShell text served for one token, with the URL and hashes baked in."""
base = (baseurl if baseurl is not None else bootstrap_base_url()).rstrip('/')
hashes = siteagent.staged_binary_hashes()
return _SCRIPT_TEMPLATE.format(
packageurl=json.dumps('{0}{1}?t={2}'.format(base, PACKAGE_PATH,
secret)),
agentsha=json.dumps(hashes.get('agent', '')),
traysha=json.dumps(hashes.get('tray', '')),
forcevariable=FORCE_VARIABLE,
waitseconds=120,
)
# ---------------------------------------------------------------------------
# Admin API
# ---------------------------------------------------------------------------
def _username():
from flask_jwt_extended import current_user
return getattr(current_user, 'username', None)
@endpoints_bp.route('/siteagent/bootstrap/tokens', methods=['POST'])
@require_role('admin')
def create_bootstrap_token():
"""Mint one short-lived bootstrap token. The secret is returned ONCE."""
payload = request.get_json(silent=True) or {}
try:
hours = int(payload.get('hours', DEFAULT_HOURS))
maxuses = int(payload.get('maxuses', DEFAULT_MAX_USES))
except (TypeError, ValueError):
return error_response(ErrorCodes.VALIDATION_ERROR,
'hours and maxuses must be whole numbers',
http_code=400)
if not MINIMUM_HOURS <= hours <= MAXIMUM_HOURS:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'hours must be between {0} and {1}'.format(MINIMUM_HOURS,
MAXIMUM_HOURS),
http_code=400)
if not MINIMUM_USES <= maxuses <= MAXIMUM_USES:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'maxuses must be between {0} and {1}'.format(MINIMUM_USES,
MAXIMUM_USES),
http_code=400)
secret = TOKEN_PREFIX + secrets.token_urlsafe(32)
now = _naive_utcnow()
row = EndpointBootstrapToken(
tokenhash=_hash_token(secret),
tokenprefix=secret[:DISPLAY_PREFIX_LENGTH],
createdat=now,
createdby=_username(),
expiresat=now + timedelta(hours=hours),
maxuses=maxuses,
usecount=0,
)
db.session.add(row)
db.session.flush()
_audit('created', row,
'bootstrap token created, {0}h, {1} use(s)'.format(hours, maxuses))
db.session.commit()
return success_response(
{
'token': row.to_dict(now),
'secret': secret,
'command': bootstrap_command(secret),
'scripturl': bootstrap_script_url(secret),
},
message='Bootstrap token created. The command is shown once.',
http_code=201)
@endpoints_bp.route('/siteagent/bootstrap/tokens', methods=['GET'])
@require_role('admin')
def list_bootstrap_tokens():
"""Every bootstrap token, newest first. Never a secret."""
now = _naive_utcnow()
rows = (EndpointBootstrapToken.query
.order_by(EndpointBootstrapToken.tokenid.desc())
.limit(LIST_LIMIT).all())
return success_response({'tokens': [row.to_dict(now) for row in rows]})
@endpoints_bp.route('/siteagent/bootstrap/tokens/<int:tokenid>/revoke',
methods=['POST'])
@require_role('admin')
def revoke_bootstrap_token(tokenid):
"""Kill a bootstrap token now. Already-revoked is a no-op, not an error."""
row = db.session.get(EndpointBootstrapToken, tokenid)
if row is None:
return error_response(ErrorCodes.NOT_FOUND, 'not found', http_code=404)
now = _naive_utcnow()
if row.revokedat is None:
row.revokedat = now
_audit('updated', row, 'bootstrap token revoked')
db.session.commit()
return success_response({'token': row.to_dict(now)},
message='Bootstrap token revoked')
# ---------------------------------------------------------------------------
# Token-authenticated: no login, the token in ?t= is the whole authorization
# ---------------------------------------------------------------------------
@endpoints_bp.route('/siteagent/bootstrap.ps1', methods=['GET'])
def serve_bootstrap_script():
"""The PowerShell the one-line command downloads, runs and deletes.
The same URL is the UI's "Download .ps1" link, for a tech who would rather
save the file and run it than paste a command.
Fetching the script does NOT spend a use: a tech who pastes the command
twice, or a shell that retries, must not burn the budget the package
download is metered by.
"""
if _ratelimited():
return _rate_limit_response()
row = _resolve_token()
if row is None:
return _not_found()
secret = (request.args.get('t') or '').strip()
_audit('requested', row, 'bootstrap script served')
db.session.commit()
return Response(build_bootstrap_script(secret), mimetype='text/plain')
@endpoints_bp.route('/siteagent/bootstrap/package', methods=['GET'])
def download_bootstrap_package():
"""The keyless agent package the bootstrap script installs.
Never carries a collector key, whatever the shared-key setting says: a
bootstrap PC enrolls and waits for approval. Counts as one use.
"""
if _ratelimited():
return _rate_limit_response()
row = _resolve_token()
if row is None:
return _not_found()
now = _naive_utcnow()
row.usecount = (row.usecount or 0) + 1
row.lastusedat = now
row.lastusedip = _caller_ip()[:45]
_audit('downloaded', row, 'bootstrap package, use {0} of {1}'.format(
row.usecount, row.maxuses))
db.session.commit()
buffer, warning = siteagent.build_keyless_package(bootstrap_base_url())
response = send_file(
buffer,
mimetype='application/zip',
as_attachment=True,
download_name=siteagent.package_download_name())
if warning:
response.headers[siteagent.CA_WARNING_HEADER] = warning
return response