Make main the lean product a site receives

main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.

WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.

WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.

A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:

deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.

agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.

DECISIONS THIS ENCODES, rather than defers:

- The plugin-authoring lab builds printedparts, whose manifest depends on
  employees, so it could not stay. The lab remains on the site line and main
  ships no build-along tutorial. That is a real loss for an adopting developer
  and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
  Each splits one notification into a card per employee and renders a photo,
  which needs a plugin this branch does not bundle. Seeding them would offer an
  author a type that cannot resolve anybody. A site that installs employees gets
  them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
  with Docker, because that is what a receiving site gets.

The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.

Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.
This commit is contained in:
cproudlock
2026-09-19 11:24:40 -04:00
parent cb3975049c
commit 534f179ee6
675 changed files with 115759 additions and 60239 deletions

View File

@@ -9,5 +9,10 @@ from . import siteagent # noqa: F401
from . import agentupload # noqa: F401
# Attaches the monitor catalog CRUD and resolve routes to endpoints_bp.
from . import monitors # noqa: F401
# Attaches per-PC credential enrollment, admin and migration routes.
from . import enrollment # noqa: F401
# Attaches the one-line bootstrap installer: token admin plus the two
# token-authenticated download routes.
from . import bootstrap # noqa: F401
__all__ = ['endpoints_bp']

View File

@@ -0,0 +1,293 @@
"""Agent authentication: per-PC credentials, with the shared key as fallback.
Every agent-facing endpoints route authenticates through authenticate_agent()
and, when the route names a PC, binds the caller through bind_hostname().
X-Endpoint-Credential The PC's own secret. The server derives the asset from
it; a hostname or assetid in the request that names a
different asset is refused with ASSET_MISMATCH.
X-API-Key The shared fleet key (or a collector-scoped managed
token). Accepted only while
plugin.endpoints.sharedkeyallowed is on, and never for a
PC that holds an active credential. A PC whose
credential was REVOKED falls back to the shared key,
which is how a rollback to an older agent works and how
the un-migrated fleet keeps reporting; a site that has
finished migrating turns
plugin.endpoints.revokeddeniessharedkey on and a revoked
PC is then refused until it enrolls again. A PC that
never enrolled has no credential row at all and is
never touched by either rule.
Only the sha256 of a credential is stored. The full secret exists once, in the
enroll response.
"""
import hashlib
import hmac
import re
import secrets
from datetime import datetime, timedelta, timezone
from flask import g, request
from shopdb.api import (
db, Setting, error_response, ErrorCodes, collector_authorized,
)
CREDENTIAL_HEADER = 'X-Endpoint-Credential'
CREDENTIAL_PREFIX = 'sdbe_'
# sdbe_ + 43 chars base64url (32 random bytes, unpadded).
_CREDENTIAL_PATTERN = re.compile(r'^sdbe_[A-Za-z0-9_-]{43}$')
DISPLAY_PREFIX_LENGTH = 12
SHARED_KEY_SETTING = 'plugin.endpoints.sharedkeyallowed'
AUTO_ENROLL_SETTING = 'plugin.endpoints.autoenroll'
# Off by default, which is what this site does today: revoking a credential
# drops the PC back to the shared key. Turning it on makes a revoke a real
# eviction - the PC reports nothing until it enrolls again - so it is only safe
# once the fleet is off the shared key, and it also closes the rollback path
# that revoke + blockreenroll exists for.
REVOKED_DENIES_SHARED_KEY_SETTING = 'plugin.endpoints.revokeddeniessharedkey'
CODE_CREDENTIAL_INVALID = 'CREDENTIAL_INVALID'
CODE_SHARED_KEY_REFUSED = 'SHARED_KEY_REFUSED'
CODE_ASSET_MISMATCH = 'ASSET_MISMATCH'
AUTH_CREDENTIAL = 'credential'
AUTH_SHARED_KEY = 'sharedkey'
# lastusedat is a display value, not an audit trail: one write per 5 minutes
# per PC keeps the long-poll from turning every claim into an UPDATE.
_TOUCH_INTERVAL = timedelta(minutes=5)
def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def generate_credential():
return CREDENTIAL_PREFIX + secrets.token_urlsafe(32)
def hash_credential(secret):
return hashlib.sha256(secret.encode('utf-8')).hexdigest()
def shared_key_allowed():
return Setting.get(SHARED_KEY_SETTING, default=True) is not False
def auto_enroll_enabled():
return Setting.get(AUTO_ENROLL_SETTING, default=True) is not False
def revoked_denies_shared_key():
"""True when a revoked credential also bars the PC from the shared key."""
return Setting.get(REVOKED_DENIES_SHARED_KEY_SETTING,
default=False) is True
def active_credential_for(assetid):
from ..models import EndpointCredential
if assetid is None:
return None
return (EndpointCredential.query
.filter(EndpointCredential.assetid == assetid,
EndpointCredential.revokedat.is_(None))
.first())
def shared_key_denied_for(assetid):
"""True when this asset may not report on the shared key.
An active credential always denies it: the PC has its own secret and a
shared-key call for it is either an older agent or somebody else.
A revoked credential denies it only while revokeddeniessharedkey is on.
Today it does not, on purpose: revoke is also how an admin rotates a
credential or rolls a PC back to an agent that only knows the shared key
(revoke + blockreenroll), and denying by default would silence that PC
instead. A PC that never enrolled has no credential row and is never denied,
which is what keeps the un-migrated fleet reporting.
"""
from ..models import EndpointCredential
if assetid is None:
return False
row = (EndpointCredential.query
.filter(EndpointCredential.assetid == assetid)
.first())
if row is None:
return False
if row.revokedat is None:
return True
return revoked_denies_shared_key()
def reenroll_blocked(assetid):
"""True when an admin blocked this asset from enrolling again."""
from ..models import EndpointCredential
if assetid is None:
return False
return (EndpointCredential.query
.filter(EndpointCredential.assetid == assetid,
EndpointCredential.reenrollblocked.is_(True))
.first()) is not None
def resolve_credential(secret, touch=True):
"""Asset id for an active credential, else None."""
from ..models import EndpointCredential
if not secret or not _CREDENTIAL_PATTERN.match(secret):
return None
digest = hash_credential(secret)
row = EndpointCredential.query.filter_by(credentialhash=digest).first()
if row is None or row.revokedat is not None:
return None
if not hmac.compare_digest(row.credentialhash, digest):
return None
if touch:
now = _naive_utcnow()
if row.lastusedat is None or now - row.lastusedat >= _TOUCH_INTERVAL:
row.lastusedat = now
db.session.commit()
return row.assetid
def resolve_hostname(hostname):
"""Asset id of the one computer with this hostname (case-insensitive)."""
from plugins.computers.models import Computer
hostname = (hostname or '').strip()
if not hostname:
return None
computer = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.first())
return computer.assetid if computer is not None else None
def shared_key_refused_for(hostname):
"""True when the shared key may not report for this hostname."""
if not shared_key_allowed():
return True
return shared_key_denied_for(resolve_hostname(hostname))
def credential_invalid_response():
return error_response(
CODE_CREDENTIAL_INVALID,
'Endpoint credential is unknown or revoked; enroll again',
http_code=401)
def shared_key_refused_response():
return error_response(
CODE_SHARED_KEY_REFUSED,
'The shared key is not accepted for this endpoint; use its credential',
http_code=401)
def shared_key_revoked_response():
"""Same code and status; the operator needs to know which refusal it is.
The PC holds no credential and cannot use the shared key either, so an
admin has to let it enroll again. Nothing changes for the agent, which
keeps retrying either way.
"""
return error_response(
CODE_SHARED_KEY_REFUSED,
'This endpoint had its credential revoked, so the shared key is not '
'accepted for it; let it enroll again',
http_code=401)
def asset_mismatch_response():
return error_response(
CODE_ASSET_MISMATCH,
'The request names a different endpoint than the credential',
http_code=403)
def hostname_unrecorded_response(hostname):
"""Refused, but say which of the two failures this actually is.
The credential is good and the hostname is simply on no computer record, so
there is nothing to compare it with. That is a data problem on this server -
an enrolled PC whose computer record carries no hostname - not a PC claiming
to be another one. Same code and status, because nothing changes for the
agent: it keeps retrying, and an admin has to set the hostname.
"""
return error_response(
CODE_ASSET_MISMATCH,
'No computer record carries the hostname {0}, so this credential '
'cannot be matched to it. Set the hostname on the enrolled '
'computer.'.format(hostname),
http_code=403)
def authenticate_agent():
"""Authorize an agent request. Returns None when authorized, else a response.
Sets g.agentauth ('credential' or 'sharedkey') and g.agentassetid (the
credential's asset, or None until bind_hostname resolves one).
"""
g.agentauth = None
g.agentassetid = None
secret = request.headers.get(CREDENTIAL_HEADER)
if secret:
assetid = resolve_credential(secret.strip())
if assetid is None:
return credential_invalid_response()
g.agentauth = AUTH_CREDENTIAL
g.agentassetid = assetid
return None
error = collector_authorized('endpoints')
if error is not None:
return error
if not shared_key_allowed():
return shared_key_refused_response()
g.agentauth = AUTH_SHARED_KEY
return None
def is_credential_caller():
return getattr(g, 'agentauth', None) == AUTH_CREDENTIAL
def bind_hostname(hostname):
"""The asset this request acts for. Returns (assetid, error_response).
Credential: the credential's asset; a hostname naming another asset is an
ASSET_MISMATCH, a hostname on no computer record at all is refused with the
same code but its own message, and a missing hostname is fine.
Shared key: the hostname's asset (None when absent or unknown, for the
route to answer); an enrolled PC is refused, and so is one whose credential
was revoked while revokeddeniessharedkey is on.
"""
hostname = (hostname or '').strip()
if is_credential_caller():
if hostname:
named = resolve_hostname(hostname)
if named is None:
return None, hostname_unrecorded_response(hostname)
if named != g.agentassetid:
return None, asset_mismatch_response()
return g.agentassetid, None
if not hostname:
return None, None
assetid = resolve_hostname(hostname)
if shared_key_denied_for(assetid):
if active_credential_for(assetid) is None:
return None, shared_key_revoked_response()
return None, shared_key_refused_response()
g.agentassetid = assetid
return assetid, None

View File

@@ -0,0 +1,127 @@
"""Register staged endpoint agent builds as installable Applications.
The endpoint update_app push resolves an install source from the applications
table alone - it has no knowledge of the agent version store. Without a row
there, a build can be staged and current in agentstore and still be impossible
to push, which is exactly the gap found in the fleet: no application row existed
for the agent at all.
This module bridges the two by making the UPLOAD the single source of truth.
Staging a build repackages the current agent (plus tray) and rewrites the
application row's installpath and installersha256 in the same request, so the
row cannot describe a build that is no longer current. Nothing here is
hand-maintained; an operator never edits these rows.
"""
import os
import uuid
from shopdb.api import Application, application_package_location, db
from . import agentstore
# Reserved appname for the managed agent bundle. appname is unique, which is
# what makes the upsert idempotent: repeated uploads update one row rather than
# accumulating a row per build. Changing this string strands the existing row,
# so it is a constant rather than anything configurable.
AGENT_APPNAME = 'ShopDB Endpoint Agent'
AGENT_PACKAGE_FILENAME = 'shopdb-endpoint-agent.zip'
_AGENT_DESCRIPTION = (
'Managed ShopDB endpoint agent bundle. Created and kept current '
'automatically when an agent build is staged - do not edit by hand.'
)
def _package_path(appid):
directory, _prefix = application_package_location()
return os.path.join(directory, str(appid), AGENT_PACKAGE_FILENAME)
def get_agent_application():
"""The reserved agent Application row, or None when nothing is registered.
Read-only lookup for callers that need the appid to build a delivery URL
without causing a row to spring into existence as a side effect.
"""
return Application.query.filter_by(appname=AGENT_APPNAME).first()
def _get_or_create_app():
"""Return the reserved agent Application row, creating it when absent.
Does not commit: the caller commits once, after the package is written, so
a failed write cannot leave a row advertising an installer that is not
there.
"""
app = Application.query.filter_by(appname=AGENT_APPNAME).first()
if app:
return app, False
app = Application(
appname=AGENT_APPNAME,
appdescription=_AGENT_DESCRIPTION,
# Installable so it can be pushed; hidden so a machine-managed row does
# not surface in the software catalogue alongside real licensed apps.
isinstallable=True,
ishidden=True,
)
db.session.add(app)
# Flush rather than commit: the row needs its appid to build the package
# path, but the transaction stays open so the whole registration is atomic.
db.session.flush()
return app, True
def register_current_agent_package():
"""Repackage the current agent build and point the agent app row at it.
Returns a dict describing what was registered, or None when there is no
current agent build to package. Safe to call on every upload: it is an
upsert keyed on the reserved appname, and it rewrites the package file in
place.
"""
payload, manifest = agentstore.build_current_package()
if payload is None:
return None
app, created = _get_or_create_app()
path = _package_path(app.appid)
os.makedirs(os.path.dirname(path), exist_ok=True)
# Write to a temp name and replace, so an interrupted write cannot leave a
# truncated archive at the path the row already advertises. os.replace is
# atomic within a filesystem. The temp name is unique per call so two
# concurrent pushes (a batch wave) cannot interleave writes into one file.
temppath = '{}.{}.tmp'.format(path, uuid.uuid4().hex)
try:
with open(temppath, 'wb') as handle:
handle.write(payload)
os.replace(temppath, path)
except BaseException:
try:
os.remove(temppath)
except OSError:
pass
raise
_directory, urlprefix = application_package_location()
app.installpath = '{}{}/{}'.format(
urlprefix, app.appid, AGENT_PACKAGE_FILENAME)
# The digest of the ARCHIVE, which is what the agent downloads and verifies.
# The per-binary digests inside the manifest are for the install script to
# check its unpacked contents, and are not interchangeable with this one.
app.installersha256 = manifest['sha256']
db.session.commit()
return {
'appid': app.appid,
'appname': app.appname,
'created': created,
'installpath': app.installpath,
'sha256': manifest['sha256'],
'size': manifest['size'],
'agentversion': manifest['agentversion'],
'contents': manifest['contents'],
}

View File

@@ -35,6 +35,7 @@ import json
import os
import re
import struct
import zipfile
from datetime import datetime
from flask import current_app
@@ -470,3 +471,109 @@ def set_current(binkey, version):
manifest['current'] = version
_write_manifest(binkey, manifest)
return True
def version_sha256(binkey, version):
"""The recorded digest for a staged version, or None when not recorded.
Read from the manifest rather than recomputed: store_version already
hashed the exact bytes it wrote, so re-reading the file to hash it again
would only introduce a way for the two to disagree.
"""
for entry in _read_manifest(binkey).get('versions', []):
if entry.get('version') == version:
return entry.get('sha256')
return None
# Name the upgrade payload carries inside the archive. Fixed rather than derived
# from the staged filenames so the install script can address its contents
# without being told what to expect.
PACKAGE_MEMBER_NAMES = {
'agent': 'shopdbagent.exe',
'tray': 'shopdb-endpoint-tray.exe',
}
# Fixed member timestamp so packaging the same build twice yields byte-identical
# archives. The value is arbitrary (the DOS epoch zip supports); only its
# constancy matters. Real build provenance lives in the manifest's versions and
# digests, not in a zip entry's mtime.
_PACKAGE_TIMESTAMP = (1980, 1, 1, 0, 0, 0)
def build_current_package():
"""Zip the current agent build (plus the current tray) into one archive.
Returns (bytes, manifestdict) or (None, None) when no agent build is
current. The agent is required; the tray is included only when it too has
a current staged version, because an endpoint can legitimately run the
service with no tray and half a tray upgrade is worse than none.
The archive is built IN MEMORY and returned rather than written here: the
caller decides where it lands, and nothing under the version store gets a
derived artifact that retention would then have to reason about.
The returned manifest describes what went in - version and digest per
member - so the install script can verify it unpacked the build the server
meant to send, rather than trusting the archive's filename.
"""
agentversion = current_version('agent')
if not agentversion:
return None, None
members = []
agentpath = _version_exe_path('agent', agentversion)
if not os.path.isfile(agentpath):
return None, None
members.append(('agent', agentversion, agentpath))
trayversion = current_version('tray')
if trayversion:
traypath = _version_exe_path('tray', trayversion)
if os.path.isfile(traypath):
members.append(('tray', trayversion, traypath))
contents = {}
buffer = io.BytesIO()
# ZIP_DEFLATED, not ZIP_STORED: these are ~70 MB self-contained binaries
# carrying the whole .NET runtime, and the archive crosses the wire to
# every endpoint being upgraded.
with zipfile.ZipFile(buffer, 'w', compression=zipfile.ZIP_DEFLATED) as archive:
for binkey, version, path in members:
try:
with open(path, 'rb') as handle:
data = handle.read()
except OSError:
# A staged file that cannot be read means the package would be
# silently incomplete. Refuse the whole thing: a partial upgrade
# payload is how an endpoint ends up with a new service exe and
# a stale tray.
return None, None
name = PACKAGE_MEMBER_NAMES[binkey]
# Build the entry with a FIXED timestamp. zipfile otherwise stamps
# each member with the current local time, which makes the archive
# bytes - and therefore its digest - different on every call for
# identical input. update_app pins a download to a digest, so a
# drifting hash would invalidate a package an endpoint is already
# fetching. Same reason the mode is set explicitly.
info = zipfile.ZipInfo(name, date_time=_PACKAGE_TIMESTAMP)
info.compress_type = zipfile.ZIP_DEFLATED
info.external_attr = 0o644 << 16
archive.writestr(info, data)
contents[binkey] = {
'filename': name,
'version': version,
# Prefer the recorded digest so the package agrees with the
# manifest; fall back to hashing when an older manifest entry
# predates digest recording.
'sha256': (version_sha256(binkey, version)
or hashlib.sha256(data).hexdigest()),
}
payload = buffer.getvalue()
return payload, {
'agentversion': agentversion,
'contents': contents,
'sha256': hashlib.sha256(payload).hexdigest(),
'size': len(payload),
}

View File

@@ -0,0 +1,144 @@
"""Seed the agent self-upgrade script into the ShopDB script library.
run_script only delivers bodies that live in the script library, pinned by
SHA-256. The upgrade template therefore has to exist as a real Script row with
a stored body before any upgrade can be pushed.
The template is source-controlled at
plugins/endpoints/api/templates/upgrade-shopdbagent.ps1 and seeded from there
rather than typed into the library by hand, so the reviewed script and the one
endpoints actually run are the same artifact. Editing the row's body in the UI
is pointless - the next seed overwrites it.
It lives under the plugin, not under agent/, because the Docker image copies
shopdb/, plugins/, migrations/ and scripts/ but NOT agent/. A template read
from agent/ resolves fine on a developer checkout and then fails in the
deployed container, which is the worst place to discover it.
One placeholder is rendered at seed time: the agent application's appid, which
only exists once agentregistry has created the row. Rendering happens BEFORE
the digest is computed, so the stored body and its sha256 always agree; a
substitution done after hashing would fail the agent's verification.
"""
import hashlib
import os
import uuid
from shopdb.api import Script, db, script_body_location
from . import agentregistry
# Reserved script name. Unique in the library, which makes seeding idempotent
# and keeps a hand-made script from colliding with the managed one.
UPGRADE_SCRIPT_NAME = 'ShopDB Endpoint Agent Self-Upgrade'
_UPGRADE_DESCRIPTION = (
'Managed: downloads the current published agent bundle and replaces the '
'installed agent in place, then restarts the service. Seeded from source '
'control - edits here are overwritten.'
)
# Substituted with the agent application's appid so the script can fetch its
# payload from the existing collector-authenticated app-package route.
_APPID_TOKEN = '{{AGENT_APPID}}'
_TEMPLATE_FILENAME = 'upgrade-shopdbagent.ps1'
def template_path():
return os.path.join(os.path.dirname(os.path.abspath(__file__)),
'templates', _TEMPLATE_FILENAME)
def _script_dir():
"""Where script bodies live. Resolved through the contract surface helper
so this cannot drift from the directory the download route serves from."""
directory, _prefix = script_body_location()
return directory
def render_template(appid):
"""Return the upgrade script body with the appid substituted.
Raises if the placeholder is missing: that means the template was edited
in a way that breaks the download, and seeding a script that would fetch
from a literal '{{AGENT_APPID}}' path is worse than refusing.
"""
with open(template_path(), 'r', encoding='utf-8') as handle:
body = handle.read()
if _APPID_TOKEN not in body:
raise ValueError(
'upgrade template is missing the {} placeholder; refusing to seed '
'a script that cannot resolve its package'.format(_APPID_TOKEN))
return body.replace(_APPID_TOKEN, str(appid))
def seed_upgrade_script():
"""Create or refresh the managed upgrade script row and its body.
Returns a dict describing the seeded script, or None when the agent
application row does not exist yet (nothing to point the download at).
Idempotent: unchanged input rewrites nothing and leaves the digest alone.
"""
app = agentregistry.get_agent_application()
if app is None:
return None
body = render_template(app.appid).encode('utf-8')
digest = hashlib.sha256(body).hexdigest()
script = Script.query.filter_by(scriptname=UPGRADE_SCRIPT_NAME).first()
created = script is None
if created:
script = Script(
scriptname=UPGRADE_SCRIPT_NAME,
scriptdescription=_UPGRADE_DESCRIPTION,
# SYSTEM: the script stops a service, writes into Program Files,
# and edits HKLM. None of that is available in a user context.
runcontext='system',
# Failures always report; success is opt-in. Capture it here
# because an upgrade that "succeeded" but rolled back internally
# is only diagnosable from the transcript.
captureoutput=True,
)
db.session.add(script)
db.session.flush()
# Body filename follows the library convention (script-<id>.ps1) so the
# collector download route serves it like any other script.
filename = 'script-{}.ps1'.format(script.scriptid)
path = os.path.join(_script_dir(), filename)
unchanged = (script.filename == filename and script.sha256 == digest
and os.path.isfile(path))
if not unchanged:
os.makedirs(_script_dir(), exist_ok=True)
# Temp-then-replace so an interrupted write cannot leave a truncated
# body at the path the row's digest already claims to describe.
temppath = '{}.{}.tmp'.format(path, uuid.uuid4().hex)
try:
with open(temppath, 'wb') as handle:
handle.write(body)
os.replace(temppath, path)
except BaseException:
try:
os.remove(temppath)
except OSError:
pass
raise
script.filename = filename
script.sha256 = digest
script.scriptdescription = _UPGRADE_DESCRIPTION
db.session.commit()
return {
'scriptid': script.scriptid,
'scriptname': script.scriptname,
'created': created,
'updated': not unchanged,
'sha256': digest,
'bytes': len(body),
'appid': app.appid,
}

View File

@@ -15,6 +15,12 @@ Authorization boundary (deliberate):
command family from application deploys (endpoints.deploy) so app-push rights
never imply SYSTEM-level agent replacement. That push pipe is a later slice.
Staging also keeps the pushable application row current (see agentregistry):
whenever the current build changes, the bundle is repackaged and the row's
installpath/installersha256 are rewritten. Registration is deliberately NOT
allowed to fail the upload - the binary is already staged and audited by then,
so a packaging problem is reported in the response payload rather than raised.
Uploads are capped and PE-checked so a wrong or hostile file cannot be staged.
"""
@@ -24,12 +30,47 @@ from shopdb.api import (success_response, error_response, ErrorCodes, AuditLog,
require_permission)
from .routes import endpoints_bp
from . import agentstore
from . import agentstore, agentregistry
# Hard cap on an uploaded agent binary. The real agent is a few MB; this is a
# generous ceiling that still refuses an operator who points at a giant wrong
# file before it is buffered fully into memory-staged storage.
_MAX_UPLOAD_BYTES = 64 * 1024 * 1024
# Hard cap on an uploaded agent binary. This is a generous ceiling that still
# refuses an operator who points at a giant wrong file before it is buffered
# fully into memory-staged storage.
#
# 128 MB, not 64: the self-contained single-file tray publishes at ~69 MB
# (it carries the whole .NET runtime plus WinForms), which the old 64 MB cap
# rejected outright. The upload failed on SIZE, but the operator saw only a
# version-detection failure, because the size check returns before the PE is
# ever parsed. Config.MAX_CONTENT_LENGTH must stay above this - it bounds the
# whole multipart envelope, so a cap equal to it would reject a legal file on
# the boundary bytes alone.
_MAX_UPLOAD_BYTES = 128 * 1024 * 1024
def _sync_agent_registration():
"""Repackage and re-register the current build. Returns a status dict.
Called from every route that can change which build is current, so the
pushable application row can never describe a build that is no longer
staged. That drift is the whole failure this is meant to prevent, and it
would be silent - the row would keep pointing at bytes nobody selected.
Never raises. Staging has already succeeded on disk and been audited by the
time this runs; a packaging or database problem must not turn a successful
upload into an error response and leave the operator thinking the binary
was rejected. The failure is reported in the response payload instead, so
the UI can surface it without losing the upload.
"""
try:
registration = agentregistry.register_current_agent_package()
except Exception as exc: # noqa: BLE001 - reported, not swallowed
db.session.rollback()
return {'ok': False, 'error': str(exc)}
if registration is None:
# No current agent build to package. Normal when only a tray has been
# staged: the bundle is defined by the agent, so there is nothing to
# register yet.
return {'ok': False, 'error': 'no current agent build to package'}
return {'ok': True, **registration}
@endpoints_bp.route('/siteagent/binaries/<binkey>', methods=['GET'])
@@ -69,11 +110,6 @@ def upload_agent_binary(binkey):
ErrorCodes.VALIDATION_ERROR,
'binkey must be one of: agent, tray',
http_code=400)
if not agentstore.is_valid_version(version):
return error_response(
ErrorCodes.VALIDATION_ERROR,
'version must be a dotted-numeric string such as 1.6.3',
http_code=400)
upload = request.files.get('file')
if upload is None or not upload.filename:
@@ -94,6 +130,22 @@ def upload_agent_binary(binkey):
'uploaded file is not a Windows executable',
http_code=400)
# Gate the version at the PE, not at the raw input. Product versions carry
# semver build metadata (1.5.0+abcdef...) that is not a legal directory
# name, so the operator's typed string is only a hint. The dotted-numeric
# FILE version stamped in the binary's PE resource is the authority; fall
# back to it whenever the typed value is missing or not dotted-numeric, and
# only reject when neither yields a usable version.
if not agentstore.is_valid_version(version):
detected = agentstore.read_pe_file_version(data)
if agentstore.is_valid_version(detected):
version = detected
else:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'version must be a dotted-numeric string such as 1.6.3',
http_code=400)
result = agentstore.store_version(binkey, version, data,
make_current=makecurrent)
@@ -110,10 +162,16 @@ def upload_agent_binary(binkey):
'madecurrent': makecurrent})
db.session.commit()
# Keep the pushable application row in step with what is now current, in
# the same request. Doing it here rather than at push time means the row
# and the staged build cannot disagree in between.
registration = _sync_agent_registration()
return success_response({
'binary': binkey,
'current': result['current'],
'versions': result['versions'],
'registration': registration,
})
@@ -196,10 +254,12 @@ def adopt_legacy_agent_binary(binkey):
ErrorCodes.VALIDATION_ERROR,
'the legacy binary could not be adopted',
http_code=400)
registration = _sync_agent_registration()
return success_response({
'binary': binkey,
'current': result['current'],
'versions': result['versions'],
'registration': registration,
})
@@ -224,8 +284,13 @@ def set_current_agent_binary(binkey):
ErrorCodes.NOT_FOUND,
'that version is not staged',
http_code=404)
# A rollback changes the bytes the fleet should receive; repackage so the
# application row follows the pointer instead of pinning the build the
# operator just rolled away from.
registration = _sync_agent_registration()
return success_response({
'binary': binkey,
'current': agentstore.current_version(binkey),
'versions': agentstore.list_versions(binkey),
'registration': registration,
})

View File

@@ -0,0 +1,786 @@
"""One-line bootstrap installer for the endpoint agent.
An admin mints a short-lived token in Settings > Endpoints and hands a tech one
command. On a PC with no agent, in an ELEVATED PowerShell: the command
downloads bootstrap.ps1 to a file, runs it, and deletes it again.
That installs the agent and tray with NO shared key, so the PC asks to enroll
and waits for an admin to approve it in Settings > Endpoints > Enrollment. This
is the path for a ONE-OFF PC; a managed bay keeps getting the agent from
GE-Enforce, and the generated script says so when it finds an agent already
installed.
Routes
POST /siteagent/bootstrap/tokens admin, audited; secret once
GET /siteagent/bootstrap/tokens admin; never a secret
POST /siteagent/bootstrap/tokens/<id>/revoke admin, audited
GET /siteagent/bootstrap.ps1 token only; not a use
GET /siteagent/bootstrap/package token only; one use
The two token-authenticated routes take the token in the
X-Endpoint-Bootstrap-Token header, and still accept the deprecated ?t= form so
a command already pasted on a bay keeps working. They answer 404 for anything
not currently usable - unknown, malformed, expired, revoked, exhausted, or no
token presented at all - so the URL confirms nothing about which tokens exist,
nor about whether the query form is still accepted here. The one 400 is a real
query token offered at a site that has turned the query form off.
The admin UI's Download .ps1 link has to use the query form (a browser link
cannot send a header), so its URL carries a marker that tells it apart from a
bay still running a command pasted before this change. Only the bay logs the
deprecation warning, which is what the rollout counts un-migrated bays with.
Both routes are per-IP rate limited with the same fixed window shape the login
and payload-download limiters use, and both write an audit row carrying the
caller's IP.
"""
import hashlib
import json
import re
import secrets
from datetime import datetime, timedelta, timezone
from flask import Response, current_app, request, send_file
from shopdb.api import (
db, AuditLog, Setting, client_ip, error_response, success_response,
ErrorCodes, require_role,
)
from ..models import EndpointBootstrapToken
from . import siteagent
from .ratelimit import ratelimited, rate_limit_response
from .routes import endpoints_bp
# sdbb_ + 43 chars base64url (32 random bytes, unpadded). Same shape as the
# per-PC credential, different prefix so a leaked value is identifiable.
TOKEN_PREFIX = 'sdbb_'
_TOKEN_PATTERN = re.compile(r'^sdbb_[A-Za-z0-9_-]{43}$')
DISPLAY_PREFIX_LENGTH = 12
DEFAULT_HOURS = 24
DEFAULT_MAX_USES = 25
MINIMUM_HOURS = 1
MAXIMUM_HOURS = 168
MINIMUM_USES = 1
MAXIMUM_USES = 500
LIST_LIMIT = 100
SCRIPT_PATH = '/api/endpoints/siteagent/bootstrap.ps1'
PACKAGE_PATH = '/api/endpoints/siteagent/bootstrap/package'
# Where the one-line command stages the script before running it. A literal,
# not $env:TEMP: the line has to survive being pasted into an already-running
# PowerShell, which expands every $ inside the outer double quotes before the
# child process sees them.
#
# NOT C:\Windows\Temp. That directory grants Users create-file and gives
# CREATOR OWNER full control of what they create, so any standard user on the
# bay can pre-plant this exact path and wait for a tech to paste the command,
# which then runs the planted content elevated. Program Files is admin-only on
# a stock install, the command re-asserts an admin-only DACL on the directory
# it creates, and it refuses to run a staged file owned by anyone else.
COMMAND_STAGE_DIR = r'C:\Program Files\ShopDB\bootstrap'
COMMAND_SCRIPT_PATH = COMMAND_STAGE_DIR + r'\shopdb-bootstrap.ps1'
# Well-known SIDs, not names: the bays are domain-joined and a renamed or
# localized Administrators group would quietly break both the grant and the
# ownership test.
ADMINISTRATORS_SID = 'S-1-5-32-544'
SYSTEM_SID = 'S-1-5-18'
# The token the two public routes accept. A header, not ?t=: a query string
# lands verbatim in the reverse proxy and gunicorn access logs, and a live
# install token has no business sitting in a log file.
BOOTSTRAP_TOKEN_HEADER = 'X-Endpoint-Bootstrap-Token'
QUERY_TOKEN_ARG = 't'
# Where the token on this request came from. Kept apart because the rollout
# counts un-migrated bays by the deprecation warning SOURCE_QUERY logs: an
# admin clicking Download .ps1 in the browser must not land in that count, and
# a request carrying no token at all must not answer differently from one
# carrying a token nobody issued.
SOURCE_HEADER = 'header'
SOURCE_QUERY = 'query'
SOURCE_UI_LINK = 'uilink'
SOURCE_NONE = 'none'
# The marker on the admin Download .ps1 link. A hint, NOT authentication: a
# caller can add it by hand, and nothing is granted by it. What matters is the
# other direction - a one-line command generated before this change cannot
# carry it, so a request without it really is a bay to go and re-issue.
UI_LINK_ARG = 'via'
UI_LINK_VALUE = 'download'
# DEPRECATED, kept on purpose: every command already pasted on a bay, and the
# UI's "Download .ps1" link (a browser link cannot send a header), carry the
# token in the query string. Accepting it stays the default so a mid-rollout
# bay keeps working; a site that has re-issued its live commands turns it off.
# Turning it off also disables the download link.
#
# Declared in the endpoints plugin's get_settings_defaults() (so the row seeds
# boolean, not string) and flipped from Settings > Endpoints > Build site agent
# > One-line install, "Accept the token in the URL". A switch with no seed and
# no control would be inert on an air-gapped box.
QUERY_TOKEN_SETTING = 'plugin.endpoints.bootstrapquerytoken'
# Fixed-window per-IP limiter, same shape and cache extension as the login
# limiter in shopdb.core.api.auth and the GE-Enforce payload limiter. A tech
# fetching a script and a package is two hits; a scanner walking token guesses
# is thousands.
BOOTSTRAP_RATELIMIT_MAX = 30
BOOTSTRAP_RATELIMIT_WINDOW_SECONDS = 60
# The endpoint-side force switch. Set it in the same command to reinstall over
# an agent that is already there.
FORCE_VARIABLE = 'SHOPDB_BOOTSTRAP_FORCE'
_HOST_PATTERN = re.compile(r'^[A-Za-z0-9.-]+(?::[0-9]{1,5})?$')
def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def _caller_ip():
"""The caller address for the audit rows and the deprecation warning.
Read the value before trusting it to identify a bay. client_ip() returns
remote_addr and nothing else, so it is only as good as what the layer in
front put there.
Under the bundled Docker stack that layer is nginx, which sets
X-Forwarded-For, and TRUSTED_PROXY_COUNT=1 on the api service makes ProxyFix
take the hop nginx appended - so this reads the real caller. Keep that count
equal to the number of proxies actually in front: add one without bumping it
and this reads the inner proxy instead.
Behind a reverse proxy that does NOT forward, or one the app was not told to
trust, every row and every warning reads the proxy's own address and the
un-migrated-bay list identifies no bay. That is a deployment mistake this
function cannot detect, which is why the token prefix below is carried as
well.
It is a real address only where the site enabled that rewrite rule (the
installer's -ClientIpSource direct), or set TRUSTED_PROXY_COUNT for a
front proxy that sets the header itself.
"""
return client_ip(request) or 'unknown'
def _ratelimited():
"""True when this caller is over budget for the current window.
The implementation moved to .ratelimit so the enrollment route shares it
rather than carrying a second copy. Budget name, config keys and defaults
are unchanged, so the behaviour here is identical.
"""
return ratelimited('endpointbootstrapratelimit',
'ENDPOINTS_BOOTSTRAP_RATELIMIT',
BOOTSTRAP_RATELIMIT_MAX,
BOOTSTRAP_RATELIMIT_WINDOW_SECONDS)
def _rate_limit_response():
return rate_limit_response('Too many bootstrap requests. Try again later.')
def _not_found():
"""The single answer for every unusable token, so nothing is confirmed."""
return error_response(ErrorCodes.NOT_FOUND, 'not found', http_code=404)
def _safe_host(value):
host = (value or '').strip()
return host if host and _HOST_PATTERN.match(host) else None
def bootstrap_base_url():
"""Base URL baked into the generated command and script.
Prefers the configured site_base_url, which already carries the scheme and
the mount path. Falls back to the forwarded host (only when a proxy is
actually declared) or the request Host plus the script root, forced to
https - the same resolution the printer installer batch generator uses,
and for the same reason: behind a reverse proxy the app sees http on a
loopback port and its url_root drops the mount.
"""
base = (Setting.get('site_base_url') or '').strip().rstrip('/')
if base:
return base
host = None
if current_app.config.get('TRUSTED_PROXY_COUNT'):
host = _safe_host(request.headers.get('X-Forwarded-Host'))
if not host:
host = _safe_host(request.host)
root = (request.script_root or '').rstrip('/')
if not host:
return root
return 'https://{0}{1}'.format(host, root)
def _base(baseurl=None):
resolved = baseurl if baseurl is not None else bootstrap_base_url()
return resolved.rstrip('/')
def bootstrap_script_url(secret, baseurl=None):
"""The UI's "Download .ps1" link: a plain file a tech can save and run.
Still the query-string form, because an <a download> cannot set a header.
The one-line command no longer uses this shape - see bootstrap_command.
It carries UI_LINK_ARG so the server can tell this click apart from a bay
running a pre-change command. Without that the admin's own Download link
fires the deprecation warning the rollout uses to FIND un-migrated bays,
and the count becomes noise the moment an admin mints a token.
"""
return '{0}{1}?{2}={3}&{4}={5}'.format(_base(baseurl), SCRIPT_PATH,
QUERY_TOKEN_ARG, secret,
UI_LINK_ARG, UI_LINK_VALUE)
def bootstrap_fetch_url(path, baseurl=None):
"""A token-free URL for the machine-to-machine fetches. The token rides in
BOOTSTRAP_TOKEN_HEADER so it never reaches an access log."""
return '{0}{1}'.format(_base(baseurl), path)
def bootstrap_command(secret, baseurl=None):
"""The one line a tech pastes into an elevated PowerShell.
Download to a file and run it, never `irm ... | iex`: Microsoft Defender
scores the piped-to-iex shape as Trojan:Win32/Commando.A!ml and kills the
process before a line of it runs. Every bay runs Defender.
Single quotes inside and no $ anywhere, so the one line means the same
whether it is pasted into cmd.exe or into a PowerShell prompt. The download
and the run are wrapped in try/finally so the staged file goes away even
when the script exits non-zero or the download itself fails.
Staging is the part worth being careful about, so the line creates
COMMAND_STAGE_DIR with an admin-only DACL, clears any leftover file, and
refuses (exit 5) to run a staged script that some other principal owns -
which is what a pre-planted file looks like. Well-known SIDs throughout,
and the owner may also be the elevated caller, since a file created by an
administrator is owned by that account rather than by the group.
The hardening is also CHECKED (exit 6). icacls is a native command, so it
reports failure only in an exit code: no exception is raised, and
$ErrorActionPreference never applies to it. Unchecked, an icacls that
cannot write the DACL leaves the directory inheriting whatever the parent
grants and the line downloads into it anyway - the silent version of the
hole this staging move was made to close. The same check also covers the
New-Item before it, because icacls on a directory that was never created
exits non-zero too.
Read as `Get-Variable LASTEXITCODE -ValueOnly`, not `$LASTEXITCODE`: the
whole line must stay free of `$`, or a paste into an already-running
PowerShell expands it in the PARENT and the child is handed `-ne 0` with
the parent's last exit code baked in.
"""
return (
'powershell -ExecutionPolicy Bypass -c "'
"try {{ New-Item -ItemType Directory -Force -Path '{stagedir}' "
'| Out-Null; '
"icacls '{stagedir}' /inheritance:r /grant '*{admin}:(OI)(CI)F' "
"'*{system}:(OI)(CI)F' | Out-Null; "
'if ((Get-Variable LASTEXITCODE -ValueOnly) -ne 0) '
"{{ Write-Host 'Refusing to run: could not restrict "
"{stagedir} to administrators (icacls failed). Fix the directory "
"or delete it, then run the command again.' -ForegroundColor Red; "
'exit 6 }}; '
"Remove-Item '{path}' -Force -ErrorAction SilentlyContinue; "
"Invoke-WebRequest -UseBasicParsing '{url}' "
"-Headers @{{'{header}'='{secret}'}} -OutFile '{path}'; "
"if ((Get-Acl '{path}').GetOwner("
'[Security.Principal.SecurityIdentifier]).Value -notin '
"@('{admin}','{system}',"
'[Security.Principal.WindowsIdentity]::GetCurrent().User.Value)) '
"{{ Write-Host 'Refusing to run: the staged bootstrap script is not "
"owned by an administrator.' -ForegroundColor Red; exit 5 }}; "
"& '{path}' }} finally {{ Remove-Item '{path}' -Force "
'-ErrorAction SilentlyContinue }}"'
).format(stagedir=COMMAND_STAGE_DIR,
path=COMMAND_SCRIPT_PATH,
url=bootstrap_fetch_url(SCRIPT_PATH, baseurl),
header=BOOTSTRAP_TOKEN_HEADER,
secret=secret,
admin=ADMINISTRATORS_SID,
system=SYSTEM_SID)
def _hash_token(secret):
return hashlib.sha256(secret.encode('utf-8')).hexdigest()
def _presented_token():
"""(secret, source) for the token this request carries.
Header first, query string second. The query form is deprecated but still
the only one a browser download link can use.
SOURCE_NONE when nothing was presented, which is NOT the same as an empty
query token: reporting a bare GET as 'query' made it answer 400 naming the
header, telling an anonymous caller whether this site had turned the query
form off, and contradicting the 404-for-everything-unusable rule at the top
of this module.
"""
header = (request.headers.get(BOOTSTRAP_TOKEN_HEADER) or '').strip()
if header:
return header, SOURCE_HEADER
query = (request.args.get(QUERY_TOKEN_ARG) or '').strip()
if not query:
return '', SOURCE_NONE
if request.args.get(UI_LINK_ARG) == UI_LINK_VALUE:
return query, SOURCE_UI_LINK
return query, SOURCE_QUERY
def _is_query_source(source):
"""Both query-string forms, the bay's and the admin download link's. The
setting is about a token in an access log, and the admin link puts one
there just as surely as a pasted command does."""
return source in (SOURCE_QUERY, SOURCE_UI_LINK)
def _query_token_allowed():
"""Defaults to True: today's pasted commands and the UI download link both
use the query form, and turning that off mid-rollout strands a bay."""
return bool(Setting.get(QUERY_TOKEN_SETTING, True))
def _query_token_refused():
"""Answered before any lookup, so it confirms nothing about any token."""
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Send the bootstrap token in the {0} header.'.format(
BOOTSTRAP_TOKEN_HEADER),
http_code=400)
def _note_token_source(source):
"""Audit suffix for a token that really resolved, and, for a query-string
token, one log line naming the caller and WHICH query path it was.
Only the bay warns. That warning is the rollout's un-migrated-bay counter
(see PER-PC-CREDENTIALS-ROLLOUT.md), so the admin's own Download .ps1 link
logs at info instead - otherwise every mint-and-test by an admin adds a
bay that does not exist.
Never the token itself, in either line: keeping it out of the logs is the
whole point of the header form.
"""
if source == SOURCE_UI_LINK:
current_app.logger.info(
'bootstrap script fetched by %s from the admin Download .ps1 '
'link; query-string token, but the admin browser path, NOT an '
'un-migrated bay', _caller_ip())
return ' (query-string token, admin download link)'
if source == SOURCE_QUERY:
current_app.logger.warning(
'bootstrap token presented in the query string by %s with no '
'admin download marker, so this is an un-migrated bay running a '
'command pasted before the header form; that form is deprecated '
'and slated for removal, re-issue the one-line install command',
_caller_ip())
return ' (query-string token, deprecated)'
return ''
def _resolve_token(secret):
"""The usable token with this secret, else None.
Unknown, malformed, expired, revoked and exhausted all resolve to None so
the caller can answer 404 for every one of them.
"""
if not _TOKEN_PATTERN.match(secret):
return None
row = EndpointBootstrapToken.query.filter_by(
tokenhash=_hash_token(secret)).first()
if row is None or not row.isusable(_naive_utcnow()):
return None
return row
def _audit(action, row, details):
AuditLog.log(action, 'EndpointBootstrapToken', entityid=row.tokenid,
entityname=row.tokenprefix,
details='{0} from {1}'.format(details, _caller_ip()))
# ---------------------------------------------------------------------------
# The generated script
# ---------------------------------------------------------------------------
# Why the wait loop watches a LOCAL file rather than a server route: there is no
# public "am I approved yet" endpoint, and adding one would hand an anonymous
# caller a way to probe enrollment state. The agent writes credential.bin the
# moment it collects an approved credential, so the file appearing IS the
# enrollment, observed without asking the server anything.
_SCRIPT_TEMPLATE = '''# ShopDB endpoint agent bootstrap
#
# Generated by the ShopDB server for one short-lived install token. Run from an
# ELEVATED Windows PowerShell. It downloads this server's agent package, checks
# the binaries against the hashes stamped in below, installs the agent and tray
# with NO shared key, and leaves the PC waiting for an admin to approve its
# enrollment in Settings > Endpoints > Enrollment.
#
# A managed bay should get the agent from GE-Enforce instead. This is the
# one-off path.
$ErrorActionPreference = "Stop"
$BootstrapPackageUrl = {packageurl}
$BootstrapPackageHeaders = @{{ {tokenheader} = {token} }}
$BootstrapServiceName = "ShopDbEndpointAgent"
$BootstrapInstallDir = "C:\\Program Files\\ShopDB\\EndpointAgent"
$BootstrapAgentSha256 = {agentsha}
$BootstrapTraySha256 = {traysha}
$BootstrapForceVariable = "{forcevariable}"
$BootstrapWaitSeconds = {waitseconds}
$BootstrapCredentialPath = Join-Path $env:ProgramData "ShopDB\\EndpointAgent\\credential\\credential.bin"
Write-Host "== ShopDB endpoint agent bootstrap ==" -ForegroundColor Cyan
# 1. Elevation. Every later step writes HKLM and Program Files.
$bootstrapIdentity = [Security.Principal.WindowsIdentity]::GetCurrent()
$bootstrapPrincipal = New-Object Security.Principal.WindowsPrincipal($bootstrapIdentity)
if (-not $bootstrapPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) {{
Write-Host "This installer must run elevated." -ForegroundColor Red
Write-Host "Open PowerShell as administrator (right-click Windows PowerShell, Run as administrator) and paste the command again."
exit 1
}}
# 2. An agent already here means this PC is probably managed.
$bootstrapForce = $false
$bootstrapForceValue = [Environment]::GetEnvironmentVariable($BootstrapForceVariable)
if ($bootstrapForceValue -and $bootstrapForceValue -ne "0") {{ $bootstrapForce = $true }}
$bootstrapExisting = Get-Service -Name $BootstrapServiceName -ErrorAction SilentlyContinue
if ($bootstrapExisting -and -not $bootstrapForce) {{
Write-Host ("The ShopDB endpoint agent is already installed here (service " + $BootstrapServiceName + ", " + $bootstrapExisting.Status + ").") -ForegroundColor Yellow
Write-Host "A managed bay should get the agent from GE-Enforce, not from this one-line installer."
Write-Host ("To reinstall anyway, run this first in the same window, then paste the command again:")
Write-Host (" " + '$env:' + $BootstrapForceVariable + " = '1'")
exit 2
}}
$bootstrapAlreadyEnrolled = Test-Path $BootstrapCredentialPath
# 3. Download the package over TLS with normal chain validation. Nothing here
# relaxes certificate checking: a server this PC does not trust must be fixed,
# not bypassed.
try {{
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
}}
catch {{ }}
$bootstrapStamp = [DateTime]::UtcNow.ToString("yyyyMMddHHmmssfff")
$bootstrapTempDir = Join-Path $env:TEMP ("shopdb-bootstrap-" + $bootstrapStamp)
$bootstrapZipPath = Join-Path $bootstrapTempDir "package.zip"
$bootstrapExtractDir = Join-Path $bootstrapTempDir "package"
function Remove-BootstrapTemp {{
if (Test-Path $bootstrapTempDir) {{
try {{ Remove-Item -Path $bootstrapTempDir -Recurse -Force -ErrorAction Stop }}
catch {{ Write-Warning ("could not remove " + $bootstrapTempDir + "; delete it manually.") }}
}}
}}
function Test-BootstrapHash {{
param(
[Parameter(Mandatory)] [string] $Path,
[AllowEmptyString()] [string] $Expected,
[Parameter(Mandatory)] [string] $Label
)
$present = Test-Path $Path
if ([string]::IsNullOrWhiteSpace($Expected)) {{
# The server staged no such binary, so the package must not carry one.
if ($present) {{
Write-Host ($Label + " is in the package but this server staged none.") -ForegroundColor Red
return $false
}}
Write-Host (" " + $Label + ": not shipped by this server")
return $true
}}
if (-not $present) {{
Write-Host ($Label + " is missing from the downloaded package.") -ForegroundColor Red
return $false
}}
$actual = (Get-FileHash -Path $Path -Algorithm SHA256).Hash.ToLowerInvariant()
if ($actual -ne $Expected.ToLowerInvariant()) {{
Write-Host ($Label + " does not match the expected SHA256.") -ForegroundColor Red
Write-Host (" expected " + $Expected)
Write-Host (" actual " + $actual)
return $false
}}
Write-Host (" " + $Label + ": sha256 verified")
return $true
}}
try {{
New-Item -ItemType Directory -Force -Path $bootstrapTempDir | Out-Null
Write-Host "[1/4] Downloading the agent package ..."
Invoke-WebRequest -Uri $BootstrapPackageUrl -Headers $BootstrapPackageHeaders -OutFile $bootstrapZipPath -UseBasicParsing
Write-Host "[2/4] Expanding and verifying ..."
Expand-Archive -Path $bootstrapZipPath -DestinationPath $bootstrapExtractDir -Force
$bootstrapOk = $true
if (-not (Test-BootstrapHash -Path (Join-Path $bootstrapExtractDir "shopdbagent.exe") -Expected $BootstrapAgentSha256 -Label "shopdbagent.exe")) {{ $bootstrapOk = $false }}
if (-not (Test-BootstrapHash -Path (Join-Path $bootstrapExtractDir "shopdb-endpoint-tray.exe") -Expected $BootstrapTraySha256 -Label "shopdb-endpoint-tray.exe")) {{ $bootstrapOk = $false }}
if (-not $bootstrapOk) {{
Remove-BootstrapTemp
Write-Host "Refusing to install an unverified package." -ForegroundColor Red
exit 3
}}
$bootstrapInstaller = Join-Path $bootstrapExtractDir "install.ps1"
if (-not (Test-Path $bootstrapInstaller)) {{
Remove-BootstrapTemp
Write-Host "The downloaded package carries no install.ps1." -ForegroundColor Red
exit 3
}}
# No key argument: this package has no -CollectorKey parameter at all.
Write-Host "[3/4] Installing ..."
& powershell.exe -NoProfile -ExecutionPolicy Bypass -File $bootstrapInstaller
$bootstrapInstallExit = $LASTEXITCODE
}}
catch {{
Remove-BootstrapTemp
Write-Host ("Bootstrap failed: " + $_.Exception.Message) -ForegroundColor Red
exit 3
}}
Remove-BootstrapTemp
# 4. Report what actually landed. The installer's own exit code is not enough:
# a PowerShell -File run can return 0 after a terminating error, so the service
# is read back directly.
Write-Host "[4/4] Result"
$bootstrapAgentPath = Join-Path $BootstrapInstallDir "shopdbagent.exe"
$bootstrapVersion = "not installed"
if (Test-Path $bootstrapAgentPath) {{
$bootstrapVersion = (Get-Item $bootstrapAgentPath).VersionInfo.FileVersion
}}
$bootstrapService = Get-Service -Name $BootstrapServiceName -ErrorAction SilentlyContinue
$bootstrapState = "not installed"
if ($bootstrapService) {{ $bootstrapState = [string] $bootstrapService.Status }}
Write-Host (" installed version : " + $bootstrapVersion)
Write-Host (" service state : " + $bootstrapState)
if (-not $bootstrapService -or $bootstrapState -ne "Running") {{
Write-Host "The agent service is not running. The install did not complete." -ForegroundColor Red
if ($bootstrapInstallExit) {{ Write-Host (" installer exit code: " + $bootstrapInstallExit) }}
exit 4
}}
if ($bootstrapAlreadyEnrolled) {{
Write-Host "This PC already holds an endpoint credential; no approval is needed." -ForegroundColor Green
exit 0
}}
Write-Host "This PC is waiting for approval - an admin approves it in Settings > Endpoints > Enrollment." -ForegroundColor Yellow
# Watch for the credential the agent writes once an admin approves it. Purely
# local: nothing is asked of the server, and a PC still pending after the wait
# is a normal outcome, not a failure.
$bootstrapDeadline = (Get-Date).AddSeconds($BootstrapWaitSeconds)
$bootstrapEnrolled = $false
while ((Get-Date) -lt $bootstrapDeadline) {{
if (Test-Path $BootstrapCredentialPath) {{ $bootstrapEnrolled = $true; break }}
Start-Sleep -Seconds 5
}}
if ($bootstrapEnrolled) {{
Write-Host "approved and enrolled" -ForegroundColor Green
}}
else {{
Write-Host "Still pending. The agent keeps asking on its own; nothing else is needed on this PC."
}}
exit 0
'''
def build_bootstrap_script(secret, baseurl=None):
"""The PowerShell text served for one token, with the URL and hashes baked in."""
base = _base(baseurl)
hashes = siteagent.staged_binary_hashes()
return _SCRIPT_TEMPLATE.format(
packageurl=json.dumps(bootstrap_fetch_url(PACKAGE_PATH, base)),
tokenheader=json.dumps(BOOTSTRAP_TOKEN_HEADER),
token=json.dumps(secret),
agentsha=json.dumps(hashes.get('agent', '')),
traysha=json.dumps(hashes.get('tray', '')),
forcevariable=FORCE_VARIABLE,
waitseconds=120,
)
# ---------------------------------------------------------------------------
# Admin API
# ---------------------------------------------------------------------------
def _username():
from flask_jwt_extended import current_user
return getattr(current_user, 'username', None)
@endpoints_bp.route('/siteagent/bootstrap/tokens', methods=['POST'])
@require_role('admin')
def create_bootstrap_token():
"""Mint one short-lived bootstrap token. The secret is returned ONCE."""
payload = request.get_json(silent=True) or {}
try:
hours = int(payload.get('hours', DEFAULT_HOURS))
maxuses = int(payload.get('maxuses', DEFAULT_MAX_USES))
except (TypeError, ValueError):
return error_response(ErrorCodes.VALIDATION_ERROR,
'hours and maxuses must be whole numbers',
http_code=400)
if not MINIMUM_HOURS <= hours <= MAXIMUM_HOURS:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'hours must be between {0} and {1}'.format(MINIMUM_HOURS,
MAXIMUM_HOURS),
http_code=400)
if not MINIMUM_USES <= maxuses <= MAXIMUM_USES:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'maxuses must be between {0} and {1}'.format(MINIMUM_USES,
MAXIMUM_USES),
http_code=400)
secret = TOKEN_PREFIX + secrets.token_urlsafe(32)
now = _naive_utcnow()
row = EndpointBootstrapToken(
tokenhash=_hash_token(secret),
tokenprefix=secret[:DISPLAY_PREFIX_LENGTH],
createdat=now,
createdby=_username(),
expiresat=now + timedelta(hours=hours),
maxuses=maxuses,
usecount=0,
)
db.session.add(row)
db.session.flush()
_audit('created', row,
'bootstrap token created, {0}h, {1} use(s)'.format(hours, maxuses))
db.session.commit()
return success_response(
{
'token': row.to_dict(now),
'secret': secret,
'command': bootstrap_command(secret),
'scripturl': bootstrap_script_url(secret),
},
message='Bootstrap token created. The command is shown once.',
http_code=201)
@endpoints_bp.route('/siteagent/bootstrap/tokens', methods=['GET'])
@require_role('admin')
def list_bootstrap_tokens():
"""Every bootstrap token, newest first. Never a secret."""
now = _naive_utcnow()
rows = (EndpointBootstrapToken.query
.order_by(EndpointBootstrapToken.tokenid.desc())
.limit(LIST_LIMIT).all())
return success_response({'tokens': [row.to_dict(now) for row in rows]})
@endpoints_bp.route('/siteagent/bootstrap/tokens/<int:tokenid>/revoke',
methods=['POST'])
@require_role('admin')
def revoke_bootstrap_token(tokenid):
"""Kill a bootstrap token now. Already-revoked is a no-op, not an error."""
row = db.session.get(EndpointBootstrapToken, tokenid)
if row is None:
return error_response(ErrorCodes.NOT_FOUND, 'not found', http_code=404)
now = _naive_utcnow()
if row.revokedat is None:
row.revokedat = now
_audit('updated', row, 'bootstrap token revoked')
db.session.commit()
return success_response({'token': row.to_dict(now)},
message='Bootstrap token revoked')
# ---------------------------------------------------------------------------
# Token-authenticated: no login, the token is the whole authorization. It
# rides in the X-Endpoint-Bootstrap-Token header; ?t= is still accepted.
# ---------------------------------------------------------------------------
@endpoints_bp.route('/siteagent/bootstrap.ps1', methods=['GET'])
def serve_bootstrap_script():
"""The PowerShell the one-line command downloads, runs and deletes.
The same URL is the UI's "Download .ps1" link, for a tech who would rather
save the file and run it than paste a command.
Fetching the script does NOT spend a use: a tech who pastes the command
twice, or a shell that retries, must not burn the budget the package
download is metered by.
"""
if _ratelimited():
return _rate_limit_response()
secret, source = _presented_token()
if _is_query_source(source) and not _query_token_allowed():
return _query_token_refused()
row = _resolve_token(secret)
if row is None:
return _not_found()
_audit('requested', row,
'bootstrap script served' + _note_token_source(source))
db.session.commit()
return Response(build_bootstrap_script(secret), mimetype='text/plain')
@endpoints_bp.route('/siteagent/bootstrap/package', methods=['GET'])
def download_bootstrap_package():
"""The keyless agent package the bootstrap script installs.
Never carries a collector key, whatever the shared-key setting says: a
bootstrap PC enrolls and waits for approval. Counts as one use.
"""
if _ratelimited():
return _rate_limit_response()
secret, source = _presented_token()
if _is_query_source(source) and not _query_token_allowed():
return _query_token_refused()
row = _resolve_token(secret)
if row is None:
return _not_found()
now = _naive_utcnow()
row.usecount = (row.usecount or 0) + 1
row.lastusedat = now
row.lastusedip = _caller_ip()[:45]
_audit('downloaded', row, 'bootstrap package, use {0} of {1}{2}'.format(
row.usecount, row.maxuses, _note_token_source(source)))
db.session.commit()
buffer, warning = siteagent.build_keyless_package(bootstrap_base_url())
response = send_file(
buffer,
mimetype='application/zip',
as_attachment=True,
download_name=siteagent.package_download_name())
if warning:
response.headers[siteagent.CA_WARNING_HEADER] = warning
return response

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,52 @@
"""Fixed-window per-IP rate limiting for the agent-facing routes.
One implementation, several budgets. The bootstrap installer routes and the
enrollment route are both unauthenticated by design - a PC with no agent has
neither a credential nor a user session - so the only thing standing between
them and a loop is a limiter. Each caller gets its own budget name so the two
surfaces cannot spend each other's allowance.
Same shape and same cache extension as the login limiter in
shopdb.core.api.auth. Under the default SimpleCache the counter is per-process,
so with N gunicorn workers the effective budget is N times the configured
maximum; that is defence in depth rather than a hard quota, and a shared cache
backend (Redis/memcached) tightens it to a true global budget.
"""
import time
from flask import current_app, request
from shopdb.api import cache, client_ip, error_response
def caller_ip():
return client_ip(request) or 'unknown'
def ratelimited(budget, configprefix, defaultmax, defaultwindow):
"""True when this caller is over budget for the current window.
budget cache key namespace, e.g. 'endpointenrollratelimit'
configprefix config key stem, e.g. 'ENDPOINTS_ENROLL_RATELIMIT'; reads
<prefix>_ENABLED, <prefix>_MAX and <prefix>_WINDOW_SECONDS
"""
if not current_app.config.get(configprefix + '_ENABLED', True):
return False
window = current_app.config.get(configprefix + '_WINDOW_SECONDS',
defaultwindow)
maxhits = current_app.config.get(configprefix + '_MAX', defaultmax)
# Time bucket makes this a fixed window: the key rolls over at each window
# boundary, so a per-hit set() cannot turn it into a sliding window.
bucket = int(time.time() // window) if window > 0 else 0
key = '{0}:{1}:{2}'.format(budget, caller_ip(), bucket)
count = cache.get(key) or 0
if count >= maxhits:
return True
cache.set(key, count + 1, timeout=window)
return False
def rate_limit_response(message):
"""The standard 429 shape every limiter in the product answers with."""
return error_response('RATE_LIMITED', message, http_code=429)

File diff suppressed because it is too large Load Diff

View File

@@ -21,11 +21,19 @@ The generated install.ps1 writes those two registry values from the operator's
inputs; appsettings.json ships with an empty baseurl and a placeholder apikey so
the agent fails closed until provisioned. The collector key is embedded only in
the install script's default parameter and is never stored or logged here.
Per-PC credentials: the shared key is optional per build. A package built
without it carries no key at all; the agent then asks to enroll and an admin
approves it. Every package pins the server's public key (ServerPins, the
lowercase hex SHA-256 of the leaf SubjectPublicKeyInfo), optionally with a
second "next" pin for a planned certificate rotation.
"""
import hashlib
import io
import json
import os
import re
import ssl
import zipfile
@@ -34,9 +42,12 @@ from flask import request, current_app, send_file
from shopdb.api import (success_response, error_response, ErrorCodes,
require_permission)
from .routes import endpoints_bp
from .routes import endpoints_bp, _spki_sha256
from .agentauth import shared_key_allowed
from . import agentstore
_PIN_PATTERN = re.compile(r'^[0-9a-f]{64}$')
# Prebuilt, staged agent binaries. Built out of band and dropped under
# instance/siteagent/ (mirrors the adoption tool's STAGED_EXE_RELPATH model).
STAGED_AGENT_RELPATH = os.path.join('siteagent', 'shopdbagent.exe')
@@ -110,6 +121,74 @@ def _validate_pem_cert(raw):
return None
return ''.join(blocks)
def _leaf_certificate_facts(pembytes):
"""(spkisha256, isca) for the FIRST certificate in a PEM, else (None, None).
The first block is the leaf in a leaf-plus-chain file, which is the cert
the agent pins and the one Import-Certificate brings into the store.
"""
from cryptography import x509
try:
text = pembytes.decode('ascii')
except (UnicodeDecodeError, AttributeError):
return None, None
start = text.find('-----BEGIN CERTIFICATE-----')
end = text.find('-----END CERTIFICATE-----', start)
if start == -1 or end == -1:
return None, None
block = (text[start:end + len('-----END CERTIFICATE-----')] + '\n').encode('ascii')
try:
certificate = x509.load_pem_x509_certificate(block)
except ValueError:
return None, None
try:
isca = bool(certificate.extensions.get_extension_for_class(
x509.BasicConstraints).value.ca)
except x509.ExtensionNotFound:
isca = False
return _spki_sha256(block), isca
def _current_server_facts():
"""(spkisha256, isca) of this server's own leaf certificate, else (None, None)."""
path = _server_cert_path()
if not path:
return None, None
try:
with open(path, 'rb') as handle:
return _leaf_certificate_facts(handle.read())
except OSError:
return None, None
def _current_server_pin():
"""Pin of this server's own leaf certificate, or None when not found.
The SPKI pin does not depend on basicConstraints, so a CA-flagged
self-signed certificate is pinned like any other leaf.
"""
return _current_server_facts()[0]
# Shown to the admin at build time (response header and UI) when the pinned
# certificate carries basicConstraints CA:TRUE.
CA_CERT_BUILD_WARNING = (
'The server certificate has basicConstraints CA:TRUE, so install.ps1 will '
'not import it into LocalMachine\\Root. The package still pins it: agents '
'1.20 and newer trust the server by pin alone. Agents older than 1.20 and '
'PowerShell 5.1 downloads without pin support will not trust this server. '
'Regenerate the server certificate in Settings > Endpoints > Server '
'certificate (the generator does not set CA), or trust a non-CA '
'certificate on those PCs with Install-ShopDBServerCertificate.ps1.')
CA_CERT_BOTH_PINS_WARNING = (
'The uploaded CA-flagged certificate does not match this server\'s own '
'certificate, so both pins were included in ServerPins: the uploaded '
'certificate and the server\'s own certificate.')
CA_WARNING_HEADER = 'X-ShopDB-Build-Warning'
# The non-secret runtime config shipped in the package. baseurl and apikey stay
# empty/placeholder so the agent fails closed until the registry is provisioned
# by install.ps1. Cadence is stamped from the operator's input at build time.
@@ -118,6 +197,7 @@ _BASE_APPSETTINGS = {
'apikey': 'REPLACE_WITH_COLLECTOR_KEY',
'cadenceseconds': 900,
'jitterseconds': 60,
'startupjitterseconds': 30,
'adoptonfirstrun': True,
'utilitiespath': '',
'logpath': '',
@@ -136,6 +216,12 @@ _BASE_APPSETTINGS = {
'UDC.exe': 'C:\\Program Files\\UDC\\UDC.exe',
'agent.exe': 'C:\\ProgramData\\udc\\mtconnect_udc\\agent\\run.bat',
},
# Approved scheduled task paths for scheduledtask_run. Shipped empty, which
# means the agent falls back to ScheduledTaskPolicy.DefaultApprovedTaskPaths
# (the Corporate Tools set). Listing entries here REPLACES those defaults, so
# a site can narrow or widen the set without an agent rebuild. Anything not
# matched fails closed.
'approvedtaskpaths': [],
}
_README = """ShopDB endpoint agent (site build)
@@ -168,20 +254,42 @@ FILES
USAGE
1. Copy this whole folder to the target endpoint.
2. Right-click install.ps1 and Run with PowerShell as administrator. It
writes the server URL and collector key to HKLM\\SOFTWARE\\GE\\ShopDB,
{provision} to HKLM\\SOFTWARE\\GE\\ShopDB,
copies the agent into Program Files, and starts the service. It also
registers the tray to autostart for every user at logon and adds a
"ShopDB Agent" Start Menu shortcut for manual relaunch.
{enrollment}
UNINSTALL
Run uninstall-shopdbagent.ps1 from an elevated PowerShell to fully remove the
agent, tray autostart, Start Menu shortcut, and credentials.
SECURITY
install.ps1 carries a live collector key in its default -CollectorKey value.
Treat this package as a secret. The key is not recoverable from the ShopDB
server after it was generated; regenerate the key in the UI to revoke it.
"""
{security}"""
_README_SHAREDKEY = {
True: {
'provision': 'writes the server URL, server pins, and collector key',
'enrollment': '',
'security': (
' install.ps1 carries a live collector key in its default -CollectorKey value.\n'
' Treat this package as a secret. The key is not recoverable from the ShopDB\n'
' server after it was generated; regenerate the key in the UI to revoke it.\n'),
},
False: {
'provision': 'writes the server URL and server pins',
'enrollment': (
' This package carries NO shared key. After install the PC requests\n'
' enrollment; approve it in Settings > Endpoints > Enrollment.\n'),
'security': (
' This package holds no secret. Each PC receives its own credential when\n'
' an admin approves its enrollment request; an existing enrolled\n'
' credential on the PC is never touched by install.ps1.\n'),
},
}
def _readme(includesharedkey):
return _README.format(**_README_SHAREDKEY[bool(includesharedkey)])
def _staged_path(relpath):
@@ -189,7 +297,9 @@ def _staged_path(relpath):
return os.path.join(current_app.instance_path, relpath)
def _install_script(baseurl, collectorkey, cadenceseconds, certfilename=None):
def _install_script(baseurl, collectorkey, cadenceseconds, certfilename=None,
serverpins=None, includesharedkey=True,
trustcertinroot=True):
"""The install.ps1 that provisions credentials and installs the service.
The operator's server URL and collector key are stamped as the default
@@ -199,10 +309,38 @@ def _install_script(baseurl, collectorkey, cadenceseconds, certfilename=None):
certificate is bundled, certfilename names it and the script imports it into
the machine Trusted Root store so the agent's outbound TLS to a self-signed
or internal-CA server validates.
serverpins are written as REG_MULTI_SZ ServerPins. With includesharedkey
False the collector key is left out of the script entirely (no parameter,
no registry write) and the installer tells the operator to approve the
enrollment request. trustcertinroot False (a CA certificate) skips the Root
import; the script also re-checks basicConstraints on the endpoint.
"""
baseurl_literal = json.dumps(baseurl)
collectorkey_literal = json.dumps(collectorkey)
certfile_literal = json.dumps(certfilename or '')
pins_literal = '@(' + ', '.join(json.dumps(pin) for pin in (serverpins or [])) + ')'
if includesharedkey:
keyparam = ' [string] $CollectorKey = {0},\n'.format(json.dumps(collectorkey))
keycheck = (
'if ([string]::IsNullOrWhiteSpace($CollectorKey)) {\n'
' throw "CollectorKey is not set. Regenerate the key in the endpoints UI and rebuild."\n'
'}\n')
keywrite = 'Set-HklmValueBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "CollectorKey" -Value $CollectorKey\n'
sharedkeymode = 'included'
enrollnotice = ''
else:
keyparam = ''
keycheck = ''
keywrite = (
'if (-not (Test-Path (Join-Path $env:ProgramData "ShopDB\\EndpointAgent\\credential\\credential.bin"))) {\n'
' Write-Host " no shared key in this package; the agent will request enrollment." -ForegroundColor Yellow\n'
'}\n')
sharedkeymode = 'omitted'
enrollnotice = (
'if (-not (Test-Path (Join-Path $env:ProgramData "ShopDB\\EndpointAgent\\credential\\credential.bin"))) {\n'
' Write-Host " This PC will request enrollment. Approve it in '
'Settings > Endpoints > Enrollment." -ForegroundColor Yellow\n'
'}\n')
return """# install.ps1
# ShopDB endpoint agent installer. Run from an ELEVATED (admin) PowerShell on
# the endpoint. Provisions the registry credentials, copies the agent to
@@ -212,12 +350,24 @@ def _install_script(baseurl, collectorkey, cadenceseconds, certfilename=None):
[CmdletBinding()]
param(
[string] $BaseUrl = {baseurl},
[string] $CollectorKey = {collectorkey},
{keyparam} [string[]] $ServerPins = {serverpins},
[string] $InstallDir = "C:\\Program Files\\ShopDB\\EndpointAgent",
[string] $ServiceName = "ShopDbEndpointAgent",
[string] $TrayExeName = "shopdb-endpoint-tray.exe",
[string] $CertFileName = {certfile},
[switch] $NoTray
[switch] $NoTray,
# Scheduled task that runs the user-context probe INSIDE the operator's own
# logon session. Mapped drives are per logon token, so this is the only
# reliable way to collect them; see the registration block below.
[string] $UserContextTaskName = "ShopDB user-context probe",
[int] $UserContextTaskMinutes = 30,
# The account the probe task runs as. Must match the account whose profile
# the agent reads the spool from (FileLoggerProvider.ShopFloorUserName), or
# the probe writes session data the service never looks at.
[string] $UserContextTaskUser = "ShopFloor",
[switch] $NoUserContextTask
)
$ErrorActionPreference = "Stop"
@@ -232,9 +382,8 @@ if (-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administra
if ([string]::IsNullOrWhiteSpace($BaseUrl)) {{
throw "BaseUrl is not set. Pass -BaseUrl 'http://<host>:<port>' when running the script."
}}
if ([string]::IsNullOrWhiteSpace($CollectorKey)) {{
throw "CollectorKey is not set. Regenerate the key in the endpoints UI and rebuild."
}}
{keycheck}$sharedKeyMode = "{sharedkeymode}"
$trustCertInRoot = {trustcert}
# Write HKLM values to BOTH the 64-bit and 32-bit (WOW6432Node) registry views.
# The agent reads Registry64 first then Registry32 and uses the first view that
@@ -261,10 +410,106 @@ function Set-HklmValueBothViews {{
}}
}}
# Lock SOFTWARE\\GE\\ShopDB to SYSTEM and Administrators, in both views.
#
# WHY. The shared fleet key is written here and stays until every consumer is
# on a per-PC credential, and the DACL this key inherits by default grants
# BUILTIN\\Users read - so any logged-on user on any bay could read the fleet
# key, which is the exact exposure the per-PC credential migration exists to
# close. Everything that legitimately reads this key runs as SYSTEM (the agent
# service, the GE-Enforce reporters) or as an admin (the adoption tool,
# Set-ShopDBServerPins.ps1). The tray reads no registry at all - it talks to the
# service over the named pipe - so nothing loses access here.
#
# It also protects ServerPins from a standard user, which is what makes the
# registry safe to treat as authoritative for pinning.
function Protect-HklmShopdbKey {{
param([Parameter(Mandatory)] [string] $SubKey)
$rights = [System.Security.AccessControl.RegistryRights]
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64,
[Microsoft.Win32.RegistryView]::Registry32)) {{
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey(
[Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {{
$key = $base.OpenSubKey(
$SubKey,
[Microsoft.Win32.RegistryKeyPermissionCheck]::ReadWriteSubTree,
$rights::ChangePermissions -bor $rights::ReadPermissions)
if (-not $key) {{ continue }}
try {{
$acl = $key.GetAccessControl(
[System.Security.AccessControl.AccessControlSections]::Access)
# Break inheritance and DROP the inherited entries rather than
# copying them in: copying is what silently preserves the
# BUILTIN\\Users read this function exists to remove.
$acl.SetAccessRuleProtection($true, $false)
foreach ($rule in @($acl.Access)) {{ $null = $acl.RemoveAccessRule($rule) }}
foreach ($sid in @("S-1-5-18", "S-1-5-32-544")) {{
$who = New-Object System.Security.Principal.SecurityIdentifier($sid)
$acl.AddAccessRule((New-Object System.Security.AccessControl.RegistryAccessRule(
$who,
$rights::FullControl,
[System.Security.AccessControl.InheritanceFlags]::ContainerInherit,
[System.Security.AccessControl.PropagationFlags]::None,
[System.Security.AccessControl.AccessControlType]::Allow)))
}}
$key.SetAccessControl($acl)
}}
finally {{ $key.Dispose() }}
}}
finally {{ $base.Dispose() }}
}}
}}
# REG_MULTI_SZ in both views, compare-before-write so a reinstall with the same
# pins leaves the value (and its last-write time) alone.
function Set-HklmMultiStringBothViews {{
param(
[Parameter(Mandatory)] [string] $SubKey,
[Parameter(Mandatory)] [string] $Name,
[Parameter(Mandatory)] [string[]] $Value
)
$wanted = ($Value -join "`n")
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64,
[Microsoft.Win32.RegistryView]::Registry32)) {{
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey(
[Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {{
$key = $base.CreateSubKey($SubKey)
try {{
$same = $false
$existingValue = $key.GetValue($Name, $null)
if ($null -ne $existingValue -and
$key.GetValueKind($Name) -eq [Microsoft.Win32.RegistryValueKind]::MultiString) {{
$same = ((@($existingValue) -join "`n") -ceq $wanted)
}}
if ($same) {{
Write-Host " $Name unchanged ($view)"
}}
else {{
$key.SetValue($Name, [string[]] $Value, [Microsoft.Win32.RegistryValueKind]::MultiString)
Write-Host " $Name written ($view)"
}}
}}
finally {{ $key.Dispose() }}
}}
finally {{ $base.Dispose() }}
}}
}}
Write-Host "== ShopDB endpoint agent install ==" -ForegroundColor Cyan
Write-Host "BaseUrl : $BaseUrl"
Write-Host "InstallDir : $InstallDir"
Write-Host "ServiceName : $ServiceName"
Write-Host "Shared key : $sharedKeyMode"
# Per-PC credential lives under ProgramData\\ShopDB\\EndpointAgent\\credential and
# belongs to the agent. This installer never writes or deletes it, so a reinstall
# or upgrade keeps an enrolled PC enrolled.
$credentialDir = Join-Path $env:ProgramData "ShopDB\\EndpointAgent\\credential"
if (Test-Path (Join-Path $credentialDir "credential.bin")) {{
Write-Host " existing enrolled credential found; kept as is."
}}
# 0. Stop and remove any previous install so the file copy is a clean overwrite.
$existing = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
@@ -287,21 +532,65 @@ foreach ($name in @("shopdbagent", $trayBase)) {{
# 1. Provision registry credentials (both 64-bit and 32-bit views).
Write-Host "[1/6] Provisioning registry HKLM:\\SOFTWARE\\GE\\ShopDB (Registry64 + Registry32) ..."
Set-HklmValueBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "BaseUrl" -Value $BaseUrl
Set-HklmValueBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "CollectorKey" -Value $CollectorKey
{keywrite}
# Server public-key pins (current, plus next during a planned rotation). An
# empty list leaves any ServerPins already on the PC untouched.
$pinList = @($ServerPins | Where-Object {{ -not [string]::IsNullOrWhiteSpace($_) }})
if ($pinList.Count -gt 0) {{
Set-HklmMultiStringBothViews -SubKey "SOFTWARE\\GE\\ShopDB" -Name "ServerPins" -Value $pinList
}}
else {{
Write-Warning "no server pin in this package; ServerPins left as is."
}}
# Harden the key AFTER the writes: the DACL is replaced, not merged, so writing
# first and locking second keeps this correct whether or not the key pre-existed.
try {{
Protect-HklmShopdbKey -SubKey "SOFTWARE\\GE\\ShopDB"
Write-Host " HKLM\\SOFTWARE\\GE\\ShopDB restricted to SYSTEM and Administrators."
}}
catch {{
Write-Warning "could not restrict HKLM\\SOFTWARE\\GE\\ShopDB: $($_.Exception.Message)"
Write-Warning "the shared key on this PC stays readable by any logged-on user until that is fixed."
}}
# 1a. Trust the bundled server certificate, if one shipped. The agent's outbound
# TLS to a self-signed or internal-CA server fails closed until the machine
# trusts the cert. Importing into LocalMachine\\Root covers the SYSTEM service.
# No-op when the deployment uses a publicly trusted (or already-trusted internal)
# certificate, in which case no cert is bundled.
# Kept for agents older than 1.20 and PowerShell download paths that do not pin.
# Only a leaf is trusted: a CA in Root would trust everything it signs.
if (-not [string]::IsNullOrWhiteSpace($CertFileName)) {{
$certPath = Join-Path $sourceDir $CertFileName
if (Test-Path $certPath) {{
Write-Host " trusting bundled server certificate $CertFileName ..."
try {{
Import-Certificate -FilePath $certPath -CertStoreLocation Cert:\\LocalMachine\\Root | Out-Null
$certIsCa = -not $trustCertInRoot
if (-not $certIsCa) {{
try {{
$probe = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($certPath)
foreach ($ext in $probe.Extensions) {{
if ($ext -is [System.Security.Cryptography.X509Certificates.X509BasicConstraintsExtension] -and
$ext.CertificateAuthority) {{ $certIsCa = $true }}
}}
}}
catch {{ }}
}}
if ($certIsCa) {{
Write-Warning "server certificate $CertFileName is a CA certificate; not importing it into LocalMachine\\Root."
if ($pinList.Count -gt 0) {{
Write-Warning "ServerPins were written: agents 1.20 and newer trust the server by pin alone."
}}
Write-Warning "Agents older than 1.20, and PowerShell 5.1 downloads without pin support, will NOT trust this server."
Write-Warning "Fix: regenerate the server certificate with the ShopDB generator (Settings > Endpoints > Server certificate; it does not set CA),"
Write-Warning "or have an admin trust a non-CA server certificate on this PC with Install-ShopDBServerCertificate.ps1."
}}
else {{
Write-Host " trusting bundled server certificate $CertFileName ..."
try {{
Import-Certificate -FilePath $certPath -CertStoreLocation Cert:\\LocalMachine\\Root | Out-Null
}}
catch {{ Write-Warning "could not import server certificate: $($_.Exception.Message)" }}
}}
catch {{ Write-Warning "could not import server certificate: $($_.Exception.Message)" }}
}}
else {{
Write-Warning "server certificate $CertFileName named but not found beside install.ps1."
@@ -362,10 +651,175 @@ if (-not $NoTray) {{
}}
catch {{ Write-Warning "could not create Start Menu shortcut: $($_.Exception.Message)" }}
# Launch the tray once now so it appears without waiting for the next
# logon. Single-instance guard makes a duplicate launch a no-op.
try {{ Start-Process -FilePath $installedTray -WorkingDirectory $InstallDir }}
catch {{ Write-Warning "could not start tray: $($_.Exception.Message)" }}
# Deliberately NOT launched here. This installer runs elevated, so
# Start-Process would hand the tray an ADMIN token, and drive letters
# are per logon token: the probe would then enumerate the installing
# admin's drive table, not the operator's. Measured on a live endpoint,
# the same two shares reported Unavailable from the elevated shell and
# OK from the operator's own session, in both Win32_NetworkConnection
# and Get-SmbMapping. Launching via explorer.exe does not fix this
# either - it de-elevates but does NOT change account.
#
# The Run key above is the correct autostart: it fires at a genuine
# logon, in the operator's own session. Collection no longer depends on
# the tray at all - the scheduled task below owns it.
Write-Host " tray will start at next logon (not launched from this elevated install)."
}}
}}
# 2b. Register the user-context probe as a scheduled task owned by the SHOPFLOOR
# USER. This is what makes mapped drives, user listeners and idle time
# collectable: it is the only mechanism that starts a process in the operator's
# own session with the operator's own token.
#
# RunLevel Limited is load-bearing, not a hardening nicety - the unelevated
# token is precisely what sees the operator's mapped drives.
#
# UserId, NOT GroupId. A group-principal task is registered against the group
# rather than against any one account, and a member of that group cannot invoke
# it on demand: Start-ScheduledTask from the ShopFloor session returns access
# denied, so the task only ever ran on its own trigger. That is exactly the bug
# where "Refresh Status" appeared to do nothing. Registering to the specific
# account makes the task startable by that account, which is what both the tray
# and any manual retry depend on. The group form is kept only as a fallback for
# a host where the account cannot be resolved - it still collects on a timer,
# it just cannot be kicked on demand.
$taskState = "skipped"
if (-not $NoUserContextTask) {{
Write-Host " registering user-context probe task ..."
$taskAgentExe = Join-Path $InstallDir "shopdbagent.exe"
$taskRegistered = $false
$taskDetail = "no principal accepted"
try {{
$taskAction = New-ScheduledTaskAction -Execute $taskAgentExe -Argument "--collect-usercontext"
# AtLogOn covers the normal case. The repeating trigger matters just as
# much: AtLogOn does NOT fire for a session that is already signed in,
# which is every install onto a machine someone is currently using.
$taskTriggers = @(New-ScheduledTaskTrigger -AtLogOn)
if ($UserContextTaskMinutes -gt 0) {{
$taskTriggers += New-ScheduledTaskTrigger -Once -At (Get-Date) -RepetitionInterval (New-TimeSpan -Minutes $UserContextTaskMinutes)
}}
$taskSettings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -StartWhenAvailable -ExecutionTimeLimit (New-TimeSpan -Minutes 5)
# Try the machine-qualified account first, then the bare name. Interactive
# LogonType so the task joins the existing interactive session instead of
# creating a separate one with its own (empty) drive table.
$taskCandidates = @(
@{{ Kind = "user"; Id = "$env:COMPUTERNAME\\$UserContextTaskUser" }},
@{{ Kind = "user"; Id = $UserContextTaskUser }},
@{{ Kind = "group"; Id = "S-1-5-32-545" }},
@{{ Kind = "group"; Id = "BUILTIN\\Users" }}
)
foreach ($candidate in $taskCandidates) {{
try {{
if ($candidate.Kind -eq "user") {{
$taskPrincipal = New-ScheduledTaskPrincipal -UserId $candidate.Id -LogonType Interactive -RunLevel Limited -ErrorAction Stop
}}
else {{
$taskPrincipal = New-ScheduledTaskPrincipal -GroupId $candidate.Id -RunLevel Limited -ErrorAction Stop
}}
Register-ScheduledTask -TaskName $UserContextTaskName -Action $taskAction -Trigger $taskTriggers -Principal $taskPrincipal -Settings $taskSettings -Force -ErrorAction Stop | Out-Null
$taskRegistered = $true
$taskDetail = "$($candidate.Kind):$($candidate.Id)"
break
}}
catch {{ $taskDetail = $_.Exception.Message }}
}}
}}
catch {{ $taskDetail = "could not build the task: $($_.Exception.Message)" }}
if ($taskRegistered) {{
# Read the task back rather than trusting Register-ScheduledTask to have
# returned cleanly: policy or a management agent can rewrite or drop a
# task after registration, and a task that exists but runs elevated
# looks healthy in the scheduler UI while producing wrong data.
$verified = Get-ScheduledTask -TaskName $UserContextTaskName -ErrorAction SilentlyContinue
if (-not $verified) {{
$taskState = "UNVERIFIED:task not found after registration"
Write-Warning " user-context task registered but cannot be read back."
}}
elseif ([string] $verified.Principal.RunLevel -eq "Highest") {{
$taskState = "UNVERIFIED:elevated"
Write-Warning " user-context task runs ELEVATED; mapped drives would be collected from the wrong token."
}}
elseif ($verified.Principal.GroupId) {{
# Registered, will collect on its triggers, but cannot be started on
# demand by the operator - so "Refresh Status" will fall back to the
# timer. Surfaced rather than reported as ok, because the symptom
# (stale session data that never responds to a refresh) is otherwise
# indistinguishable from a broken probe.
$taskState = "DEGRADED:group principal $($verified.Principal.GroupId)"
Write-Warning " user-context task fell back to a GROUP principal; on-demand refresh will not work, only the $UserContextTaskMinutes-minute interval."
}}
else {{
$taskState = "ok:$($verified.Principal.UserId)/$($verified.Principal.RunLevel)"
Write-Host " user-context task registered and verified: $taskState"
}}
# AtLogOn will not fire for an already-signed-in session, so kick it once
# rather than leaving a fresh install blind until the repeat interval.
try {{
Start-ScheduledTask -TaskName $UserContextTaskName -ErrorAction Stop
Write-Host " user-context task started once for the current session."
}}
catch {{ Write-Warning " could not start the task now: $($_.Exception.Message); it runs at next logon or interval." }}
}}
else {{
# Not fatal: the service still checks in, only the session-scoped fields
# are missing, and the ledger records why.
$taskState = "FAILED:$taskDetail"
Write-Warning " could not register the user-context task: $taskDetail"
Write-Warning " mapped drives and idle time will not be collected."
}}
}}
# 2c. Start the tray in the operator's session.
#
# The Run key registered above only fires at LOGON, so installing onto a machine
# someone is already signed in to leaves no tray until they sign out and back in.
# The installer cannot start it directly - it is elevated, and the tray must run
# unelevated in the user's own session - so we borrow the same mechanism that
# solves this for the probe: a transient scheduled task with a Limited group
# principal, run once and then deleted.
#
# Registered without a trigger; it exists only to be invoked immediately. The
# tray is single-instance, so if one is somehow already running this is a no-op.
if (-not $NoTray -and (Test-Path (Join-Path $InstallDir $TrayExeName))) {{
$launchTaskName = "ShopDB tray first launch"
try {{
$launchExe = Join-Path $InstallDir $TrayExeName
$launchAction = New-ScheduledTaskAction -Execute $launchExe -WorkingDirectory $InstallDir
$launchSettings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries
$launched = $false
foreach ($groupId in @("S-1-5-32-545", "BUILTIN\\Users")) {{
try {{
$launchPrincipal = New-ScheduledTaskPrincipal -GroupId $groupId -RunLevel Limited -ErrorAction Stop
Register-ScheduledTask -TaskName $launchTaskName -Action $launchAction -Principal $launchPrincipal -Settings $launchSettings -Force -ErrorAction Stop | Out-Null
Start-ScheduledTask -TaskName $launchTaskName -ErrorAction Stop
$launched = $true
break
}}
catch {{ }}
}}
# Give the task a moment to spawn the tray before removing it: deleting a
# task that is still starting can kill the process it just launched.
if ($launched) {{
Start-Sleep -Seconds 3
Write-Host " tray started in the operator's session."
}}
else {{
Write-Warning " could not start the tray now; it will start at next logon via the Run key."
}}
# Transient by design: this is a one-shot launcher, not an autostart.
# The Run key owns autostart, and leaving this behind would mean two
# things racing to start the same single-instance tray at every logon.
try {{ Unregister-ScheduledTask -TaskName $launchTaskName -Confirm:$false -ErrorAction Stop }}
catch {{ Write-Warning " could not remove the transient launch task '$launchTaskName'; remove it manually." }}
}}
catch {{
Write-Warning " could not start the tray now: $($_.Exception.Message); it will start at next logon."
}}
}}
@@ -394,24 +848,31 @@ if (-not $serviceOk) {{
throw "Service $ServiceName did not reach Running after 5 attempts. Leaving install.ps1 in place for diagnosis; re-run once resolved."
}}
Write-Host " service is running." -ForegroundColor Green
{enrollnotice}
$trayBase2 = [System.IO.Path]::GetFileNameWithoutExtension($TrayExeName)
if (-not $NoTray -and (Test-Path (Join-Path $InstallDir $TrayExeName))) {{
$trayOk = $false
for ($attempt = 1; $attempt -le 5; $attempt++) {{
if (Get-Process -Name $trayBase2 -ErrorAction SilentlyContinue) {{ $trayOk = $true; break }}
Write-Host " tray not running yet (attempt $attempt/5); relaunching ..."
try {{ Start-Process -FilePath (Join-Path $InstallDir $TrayExeName) -WorkingDirectory $InstallDir }} catch {{ }}
# Report only; never relaunch from here. This block used to Start-Process the
# tray when it was not found running, which would hand it this installer's
# ADMIN token and reintroduce the wrong-drive-table bug. The launch task
# above is the only place the tray is started during install, and it starts
# it correctly - unelevated, in the operator's session.
$trayFound = $false
for ($attempt = 1; $attempt -le 3; $attempt++) {{
if (Get-Process -Name $trayBase2 -ErrorAction SilentlyContinue) {{ $trayFound = $true; break }}
Start-Sleep -Seconds 2
}}
if ($trayOk) {{ Write-Host " tray is running." -ForegroundColor Green }}
else {{ Write-Warning "tray did not confirm running; it will retry at next logon via the Run key." }}
if ($trayFound) {{
Write-Host " tray is running." -ForegroundColor Green
}}
else {{
Write-Warning "tray did not confirm running; it will start at next logon via the Run key."
}}
}}
# 5. Preserve the install artifacts (minus the secret-bearing install.ps1) under
# ProgramData so the uninstaller and appsettings/cert are available later, then
# scrub install.ps1 from wherever it was run. install.ps1 carries the live
# collector key in its default parameter, so it must not linger on the endpoint.
# scrub install.ps1 from wherever it was run. A shared-key build carries the live
# key in its default parameter, so it must not linger on the endpoint.
Write-Host "[5/6] Archiving install artifacts and removing install.ps1 ..."
$dataDir = Join-Path $env:ProgramData "ShopDB\\EndpointAgent"
New-Item -ItemType Directory -Force -Path $dataDir | Out-Null
@@ -427,11 +888,34 @@ $uninstaller = Join-Path $dataDir "uninstall-shopdbagent.ps1"
$installSelf = Join-Path $sourceDir "install.ps1"
if (Test-Path $installSelf) {{
try {{ Remove-Item -Path $installSelf -Force -ErrorAction Stop; Write-Host " removed install.ps1 (carried the collector key)." }}
catch {{ Write-Warning "could not remove install.ps1 - delete it manually, it contains the collector key: $($_.Exception.Message)" }}
try {{ Remove-Item -Path $installSelf -Force -ErrorAction Stop; Write-Host " removed install.ps1." }}
catch {{ Write-Warning "could not remove install.ps1 - delete it manually: $($_.Exception.Message)" }}
}}
Write-Host "[6/6] Done." -ForegroundColor Green
# Append one line per install, including reinstalls and repeat installs of the
# same version. The user-context task state is recorded because it is the single
# point of failure for session-scoped collection: if mapped drives go missing
# later, this answers "was the task ever registered on this host, and as what"
# without needing the endpoint back. The ledger is a record, not a gate - a
# failure to append must never fail an otherwise-good install.
try {{
$ledgerDir = Join-Path $env:ProgramData "ShopDB\\Logs"
New-Item -ItemType Directory -Force -Path $ledgerDir | Out-Null
$ledgerPath = Join-Path $ledgerDir "shopdbagent-installs.log"
$ledgerAgentVersion = "unknown"
if (Test-Path $installedAgent) {{ $ledgerAgentVersion = (Get-Item $installedAgent).VersionInfo.FileVersion }}
$ledgerTrayVersion = "not-installed"
$ledgerTrayPath = Join-Path $InstallDir $TrayExeName
if (Test-Path $ledgerTrayPath) {{ $ledgerTrayVersion = (Get-Item $ledgerTrayPath).VersionInfo.FileVersion }}
$ledgerLine = "{{0}} host={{1}} installer={{2}} service={{3}} tray={{4}} dir={{5}} usercontexttask={{6}} sharedkey={{7}} serverpins={{8}}" -f (Get-Date -Format "yyyy-MM-dd HH:mm:ss zzz"), $env:COMPUTERNAME, $env:USERNAME, $ledgerAgentVersion, $ledgerTrayVersion, $InstallDir, $taskState, $sharedKeyMode, $pinList.Count
Add-Content -Path $ledgerPath -Value $ledgerLine -Encoding UTF8
Write-Host "Install ledger: $ledgerPath"
Write-Host " $ledgerLine" -ForegroundColor DarkGray
}}
catch {{ Write-Warning "could not append to the install ledger: $($_.Exception.Message)" }}
if (Test-Path $uninstaller) {{
Write-Host "Uninstaller: $uninstaller" -ForegroundColor Cyan
Write-Host " To remove the agent, run that script from an elevated PowerShell."
@@ -441,11 +925,120 @@ else {{
}}
""".format(
baseurl=baseurl_literal,
collectorkey=collectorkey_literal,
keyparam=keyparam,
keycheck=keycheck,
keywrite=keywrite,
sharedkeymode=sharedkeymode,
enrollnotice=enrollnotice,
serverpins=pins_literal,
trustcert='$true' if trustcertinroot else '$false',
certfile=certfile_literal,
)
def build_package_zip(baseurl, collectorkey, cadenceseconds, appsettings,
certfilename=None, cert_pem=None, certpath=None,
serverpins=None, includesharedkey=True,
trustcertinroot=True):
"""Assemble the agent package ZIP in memory.
The single place the archive is laid out, so the admin download route and
the token-authenticated bootstrap download cannot drift apart in what they
ship. Callers own validation, pin resolution and the appsettings they pass.
"""
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, 'w', zipfile.ZIP_DEFLATED) as archive:
archive.writestr('appsettings.json',
json.dumps(appsettings, indent=2) + '\n')
archive.writestr('install.ps1',
_install_script(baseurl.rstrip('/'), collectorkey,
cadenceseconds, certfilename,
serverpins=serverpins,
includesharedkey=includesharedkey,
trustcertinroot=trustcertinroot))
archive.writestr('README.txt', _readme(includesharedkey))
if os.path.isfile(UNINSTALLER_PATH):
archive.write(UNINSTALLER_PATH, 'uninstall-shopdbagent.ps1')
agent = agentstore.current_path('agent')
if agent and os.path.isfile(agent):
archive.write(agent, 'shopdbagent.exe')
tray = agentstore.current_path('tray')
if tray and os.path.isfile(tray):
archive.write(tray, 'shopdb-endpoint-tray.exe')
if cert_pem:
archive.writestr('server.crt', cert_pem)
elif certpath:
archive.write(certpath, 'server.crt')
buffer.seek(0)
return buffer
def package_download_name():
"""Versioned download filename for the package, or the plain one."""
agentversion = agentstore.current_version('agent')
return ('shopdb-endpoint-agent-{0}.zip'.format(agentversion)
if agentversion else 'shopdb-endpoint-agent.zip')
# Filenames the package ships the staged binaries under, keyed by store binkey.
STAGED_EXE_NAMES = {
'agent': 'shopdbagent.exe',
'tray': 'shopdb-endpoint-tray.exe',
}
def staged_binary_hashes():
"""{binkey: lowercase sha256} for every staged binary, '' when none.
The bootstrap script bakes these in and re-checks the downloaded ZIP
against them on the endpoint, so a package swapped in transit is refused
before anything is executed. Hashed from the file rather than read out of
the store manifest so a legacy flat-file staging is covered too.
"""
hashes = {}
for binkey in STAGED_EXE_NAMES:
path = agentstore.current_path(binkey)
digest = ''
if path and os.path.isfile(path):
hasher = hashlib.sha256()
with open(path, 'rb') as handle:
for chunk in iter(lambda: handle.read(65536), b''):
hasher.update(chunk)
digest = hasher.hexdigest()
hashes[binkey] = digest
return hashes
def build_keyless_package(baseurl, cadenceseconds=900):
"""The package the bootstrap installer downloads: never a collector key.
Pins this server exactly as the admin builder does with no upload - the
server's own leaf certificate - and bundles that certificate when the app
can read it, which is the self-signed topology where the endpoint would
otherwise fail its own TLS validation.
"""
appsettings = dict(_BASE_APPSETTINGS)
appsettings['cadenceseconds'] = cadenceseconds
appsettings['apikey'] = ''
certpath = _server_cert_path()
certfilename = 'server.crt' if certpath else None
currentpin, pinisca = _current_server_facts()
serverpins = [pin for pin in (currentpin,) if pin]
appsettings['serverpins'] = serverpins
buffer = build_package_zip(
baseurl, '', cadenceseconds, appsettings,
certfilename=certfilename, certpath=certpath,
serverpins=serverpins, includesharedkey=False,
trustcertinroot=not pinisca)
warning = CA_CERT_BUILD_WARNING if (currentpin and pinisca) else None
return buffer, warning
def _siteagent_status_payload():
"""Whether a prebuilt agent binary is staged for packaging.
@@ -458,6 +1051,8 @@ def _siteagent_status_payload():
'agentstaged': agent_staged,
'traystaged': tray_staged,
'certavailable': _server_cert_path() is not None,
'serverpin': _current_server_pin(),
'sharedkeyallowed': shared_key_allowed(),
'agentcurrent': agentstore.current_version('agent'),
'traycurrent': agentstore.current_version('tray'),
'agentversions': agentstore.list_versions('agent'),
@@ -480,7 +1075,12 @@ def package_siteagent():
"""Bundle the site agent as a downloadable ZIP.
Accepts either JSON or multipart/form-data with fields baseurl,
collectorkey, cadenceseconds, and bundlecert. A multipart request may also
collectorkey, cadenceseconds, bundlecert, includesharedkey, and
nextserverpin. includesharedkey defaults to the site setting
plugin.endpoints.sharedkeyallowed; when off, collectorkey is ignored and
nothing in the package carries it. nextserverpin (64 lowercase hex) is
shipped beside the current server pin for a planned certificate rotation.
A multipart request may also
carry a certfile upload: an operator-supplied .crt/.pem that is validated as
X.509 and bundled so the installer trusts it as a root on the endpoint. This
is the reverse-proxy path where the app cannot see its own TLS cert. When no
@@ -490,7 +1090,9 @@ def package_siteagent():
Returns an in-memory ZIP with the staged shopdbagent.exe (and tray when
present), a non-secret appsettings.json, install.ps1, and README.txt. The
collector key is embedded only in install.ps1 as its default parameter and
is never stored or logged.
is never stored or logged. When the pinned certificate is CA-flagged the
response carries an X-ShopDB-Build-Warning header explaining that only
pinning (agents 1.20 and newer) will trust the server.
"""
if request.form or request.files:
source = request.form
@@ -503,11 +1105,25 @@ def package_siteagent():
collectorkey = (source.get('collectorkey') or '').strip()
cadenceseconds = source.get('cadenceseconds', 900)
bundlecert = str(source.get('bundlecert', '')).lower() in ('1', 'true', 'yes', 'on')
rawinclude = source.get('includesharedkey')
if rawinclude is None or str(rawinclude).strip() == '':
includesharedkey = shared_key_allowed()
else:
includesharedkey = str(rawinclude).lower() in ('1', 'true', 'yes', 'on')
nextserverpin = (source.get('nextserverpin') or '').strip()
if not baseurl or not collectorkey:
if not includesharedkey:
collectorkey = ''
if not baseurl or (includesharedkey and not collectorkey):
return error_response(
ErrorCodes.VALIDATION_ERROR,
'baseurl and collectorkey are required',
'baseurl and collectorkey are required' if includesharedkey
else 'baseurl is required',
http_code=400)
if nextserverpin and not _PIN_PATTERN.match(nextserverpin):
return error_response(
ErrorCodes.VALIDATION_ERROR,
'nextserverpin must be 64 lowercase hex characters',
http_code=400)
try:
@@ -525,6 +1141,8 @@ def package_siteagent():
appsettings = dict(_BASE_APPSETTINGS)
appsettings['cadenceseconds'] = cadenceseconds
if not includesharedkey:
appsettings['apikey'] = ''
# An operator-uploaded cert wins over the auto-detected server cert: it is
# the explicit reverse-proxy answer where the app cannot see its own TLS
@@ -544,40 +1162,62 @@ def package_siteagent():
certfilename = 'server.crt' if (cert_pem or certpath) else None
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, 'w', zipfile.ZIP_DEFLATED) as archive:
archive.writestr('appsettings.json',
json.dumps(appsettings, indent=2) + '\n')
archive.writestr('install.ps1',
_install_script(baseurl.rstrip('/'), collectorkey,
cadenceseconds, certfilename))
archive.writestr('README.txt', _README)
if os.path.isfile(UNINSTALLER_PATH):
archive.write(UNINSTALLER_PATH, 'uninstall-shopdbagent.ps1')
agent = agentstore.current_path('agent')
if agent and os.path.isfile(agent):
archive.write(agent, 'shopdbagent.exe')
tray = agentstore.current_path('tray')
if tray and os.path.isfile(tray):
archive.write(tray, 'shopdb-endpoint-tray.exe')
# Pin the leaf the server presents: the bundled/uploaded leaf, else this
# server's own certificate. The SPKI pin ignores basicConstraints, so a
# CA-flagged self-signed cert is still pinned; the CA flag only gates the
# Root import.
trustcertinroot = True
currentpin = None
pinisca = False
if certfilename:
if cert_pem:
archive.writestr('server.crt', cert_pem)
elif certpath:
archive.write(certpath, 'server.crt')
bundledbytes = cert_pem.encode('ascii')
else:
with open(certpath, 'rb') as handle:
bundledbytes = handle.read()
currentpin, bundledisca = _leaf_certificate_facts(bundledbytes)
trustcertinroot = not bundledisca
pinisca = bool(bundledisca)
# An uploaded CA-flagged cert may be the server's own self-signed cert or an
# internal CA root sent by mistake. Ship the server's own pin beside it so
# the agent (which accepts any listed pin) works either way.
serverownpin = None
if cert_pem and pinisca:
serverownpin = _current_server_pin()
if serverownpin == currentpin:
serverownpin = None
if currentpin is None:
currentpin, pinisca = _current_server_facts()
warnings = [CA_CERT_BUILD_WARNING] if (currentpin and pinisca) else []
if serverownpin:
warnings.append(CA_CERT_BOTH_PINS_WARNING)
if nextserverpin and currentpin is None:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'nextserverpin needs a current server pin; bundle or upload the '
'server leaf certificate',
http_code=400)
serverpins = [pin for pin in (currentpin, serverownpin, nextserverpin) if pin]
serverpins = list(dict.fromkeys(serverpins))
appsettings['serverpins'] = serverpins
buffer = build_package_zip(
baseurl, collectorkey, cadenceseconds, appsettings,
certfilename=certfilename, cert_pem=cert_pem, certpath=certpath,
serverpins=serverpins, includesharedkey=includesharedkey,
trustcertinroot=trustcertinroot)
buffer.seek(0)
# Stamp the staged agent version into the download filename so a DT operator
# can see which agent build a package carries at a glance. This is cosmetic:
# install.ps1 finds the exe by its fixed name inside the ZIP, so the download
# name never affects the install. Falls back to the unversioned name when no
# managed version is current (only a legacy flat file is staged).
agentversion = agentstore.current_version('agent')
download_name = ('shopdb-endpoint-agent-{0}.zip'.format(agentversion)
if agentversion else 'shopdb-endpoint-agent.zip')
return send_file(
download_name = package_download_name()
response = send_file(
buffer,
mimetype='application/zip',
as_attachment=True,
download_name=download_name)
if warnings:
response.headers[CA_WARNING_HEADER] = ' '.join(warnings)
return response

View File

@@ -0,0 +1,658 @@
# upgrade-shopdbagent.ps1
# ShopDB Endpoint Agent SELF-UPGRADE template. Delivered to an endpoint by the
# run_script command and executed by the agent as SYSTEM.
#
# WHY THIS TAKES NO PARAMETERS
# run_script executes "powershell.exe -File <script>" with no argument list, so
# there is no channel to pass BaseUrl/CollectorKey in. That is not a problem
# here: an upgrade by definition targets an endpoint that is ALREADY installed
# and therefore already holds its own config. This script READS that config and
# never rewrites it - the values stay exactly as provisioned, so an upgrade can
# not silently repoint an endpoint at a different server or key. A fresh
# install is still install-shopdbagent.ps1's job.
#
# WHY IT RE-LAUNCHES ITSELF DETACHED
# The agent process is what runs this script, and the upgrade must stop that
# same agent to release the lock on shopdbagent.exe (single-file self-contained
# exes are held open while running). If the script stayed a child of the agent,
# stopping the service would kill the script mid-copy and leave the install
# directory half-written. The first pass therefore copies itself to a temp
# location, relaunches detached, and exits immediately so the agent sees a
# clean, fast result. All real work happens in the detached second pass.
#
# REPORTING
# Success is NOT reported through the command result - by the time the upgrade
# succeeds, the agent that would have reported it has been restarted. The new
# version is confirmed on the next check-in. This script's own transcript is
# the local record; see $LogFile below.
#
# AUTH AND PINS
# The download authenticates the way the agent does: the per-PC credential from
# the agent's SYSTEM-only credential.bin when this PC has enrolled, otherwise
# the shared CollectorKey. When ServerPins is set (registry, or serverpins in
# appsettings.json) the server's TLS key must match a pin or nothing is
# downloaded. A 1.19 agent upgrading to 1.20 has no credential yet and uses the
# shared key; the new agent enrolls itself after it starts.
[CmdletBinding()]
param(
# Set only when the script relaunches itself. Operators never pass this.
[switch] $Detached
)
$ErrorActionPreference = 'Stop'
$ServiceName = 'ShopDbEndpointAgent'
$InstallDir = 'C:\Program Files\ShopDB\EndpointAgent'
$TrayExeName = 'shopdb-endpoint-tray.exe'
$AgentExeName = 'shopdbagent.exe'
$StateDir = Join-Path $env:ProgramData 'ShopDB\EndpointAgent'
$WorkDir = Join-Path $StateDir 'upgrade'
$LogDir = Join-Path $StateDir 'logs'
$LogFile = Join-Path $LogDir ('upgrade-{0}.log' -f (Get-Date -Format 'yyyyMMdd-HHmmss'))
# Server-relative path of the managed agent bundle. This reuses the EXISTING
# agent-facing app-package route rather than adding a parallel download: it
# already authenticates with the collector key and already resolves an
# application's installpath, which is precisely what upload registration
# stamped for the agent bundle. The appid is substituted by the server when
# this template is rendered, because it is assigned when the row is created.
$PackagePath = '/api/endpoints/apppackage/app/{{AGENT_APPID}}'
function Write-Log {
param([string] $Message, [string] $Level = 'INFO')
$line = '{0} [{1}] {2}' -f (Get-Date -Format 'yyyy-MM-dd HH:mm:ss'), $Level, $Message
Write-Host $line
try { Add-Content -Path $LogFile -Value $line -ErrorAction Stop } catch { }
}
# The shared credential block below logs through Log, as the reporters do.
function Log([string] $msg) { Write-Log $msg }
# ---------------------------------------------------------------------------
# Per-PC credential and server pin. Kept in step with the same block in
# Report-AssetToShopDB.ps1, Report-PrintersToShopDB.ps1 and
# upgrade-shopdbagent.ps1: each runs standalone, so the text is duplicated.
#
# Credential: the endpoint agent (1.20+) enrolls and stores its own credential
# in credential.bin, DPAPI LocalMachine, readable only by SYSTEM and
# Administrators. When present it is sent as X-Endpoint-Credential and the
# shared key is not sent. Without it the shared CollectorKey goes as X-API-Key.
#
# Pins: HKLM\SOFTWARE\GE\ShopDB ServerPins (REG_MULTI_SZ), lowercase hex
# SHA-256 of the server leaf certificate's SubjectPublicKeyInfo. When any pin
# is set the TLS handshake is accepted only if the leaf key matches one.
# ---------------------------------------------------------------------------
$ShopDbCredentialFile = Join-Path $env:ProgramData 'ShopDB\EndpointAgent\credential\credential.bin'
$ShopDbPinSource = @'
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
public static class ShopDbServerPinValidator
{
private static readonly HashSet<string> Pins = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
public static string LastPresented;
public static void Install(string[] pins)
{
Pins.Clear();
foreach (string pin in pins)
{
if (!String.IsNullOrEmpty(pin)) { Pins.Add(pin); }
}
ServicePointManager.ServerCertificateValidationCallback = Validate;
}
public static bool Validate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors errors)
{
if (Pins.Count == 0) { return errors == SslPolicyErrors.None; }
if (certificate == null) { return false; }
try
{
LastPresented = SpkiSha256Hex(certificate.GetRawCertData());
}
catch (Exception)
{
LastPresented = null;
return false;
}
return Pins.Contains(LastPresented);
}
public static bool IsPinned(string pin)
{
return !String.IsNullOrEmpty(pin) && Pins.Contains(pin);
}
// Walks the DER: Certificate > TBSCertificate > [0] version (optional),
// serialNumber, signature, issuer, validity, subject, subjectPublicKeyInfo.
public static string SpkiSha256Hex(byte[] der)
{
int length;
int position = EnterSequence(der, 0, out length);
position = EnterSequence(der, position, out length);
if (der[position] == 0xA0) { position = Skip(der, position); }
for (int field = 0; field < 5; field++) { position = Skip(der, position); }
if (der[position] != 0x30) { throw new FormatException("SubjectPublicKeyInfo not found"); }
int end = Skip(der, position);
using (SHA256 sha = SHA256.Create())
{
byte[] hash = sha.ComputeHash(der, position, end - position);
return BitConverter.ToString(hash).Replace("-", "").ToLowerInvariant();
}
}
private static int ReadLength(byte[] der, int position, out int length)
{
int first = der[position];
if (first < 0x80) { length = first; return position + 1; }
int count = first & 0x7F;
if (count == 0 || count > 4) { throw new FormatException("Unsupported DER length"); }
length = 0;
for (int i = 1; i <= count; i++) { length = (length << 8) | der[position + i]; }
return position + 1 + count;
}
private static int EnterSequence(byte[] der, int position, out int length)
{
if (der[position] != 0x30) { throw new FormatException("Expected SEQUENCE"); }
return ReadLength(der, position + 1, out length);
}
private static int Skip(byte[] der, int position)
{
int length;
int content = ReadLength(der, position + 1, out length);
return content + length;
}
}
'@
function Get-ShopDbServerPins {
# Both registry views, like CollectorKey. Anything not 64 hex chars is dropped.
$pins = @()
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64, [Microsoft.Win32.RegistryView]::Registry32)) {
try {
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {
$key = $base.OpenSubKey('SOFTWARE\GE\ShopDB')
if ($key) {
try {
foreach ($value in @($key.GetValue('ServerPins'))) {
if ($null -eq $value) { continue }
$pin = ([string]$value -replace '[\s:]', '').ToLowerInvariant()
if ($pin -match '^[0-9a-f]{64}$' -and $pins -notcontains $pin) { $pins += $pin }
}
} finally { $key.Dispose() }
}
} finally { $base.Dispose() }
} catch {}
}
return ,$pins
}
function Test-ShopDbServerPinsValue {
# True when ServerPins exists in either registry view, even empty. The
# registry then owns the pin list and appsettings pins are ignored, the
# same rule the agent follows.
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64, [Microsoft.Win32.RegistryView]::Registry32)) {
try {
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {
$key = $base.OpenSubKey('SOFTWARE\GE\ShopDB')
if ($key) {
try {
if ($key.GetValueNames() -contains 'ServerPins') { return $true }
} finally { $key.Dispose() }
}
} finally { $base.Dispose() }
} catch {}
}
return $false
}
function Enable-ShopDbServerPins([string[]]$Pins) {
# True when pinning is active. False means pins are set but could not be
# enforced; the caller must not send anything then.
if ($PSVersionTable.PSEdition -eq 'Core') {
# PowerShell 7 web cmdlets ignore ServicePointManager, so the check
# would silently not happen.
Log 'ERROR ServerPins is set; pin checking needs Windows PowerShell 5.1 (powershell.exe), not pwsh'
return $false
}
try {
if (-not ('ShopDbServerPinValidator' -as [type])) {
Add-Type -TypeDefinition $ShopDbPinSource -ErrorAction Stop
}
[ShopDbServerPinValidator]::Install($Pins)
return $true
} catch {
Log "ERROR ServerPins is set but pin checking could not be enabled: $($_.Exception.Message)"
return $false
}
}
function Get-ShopDbEndpointCredential([string]$TargetUrl) {
# Empty when there is none or it cannot be used for this URL.
if (-not (Test-Path -LiteralPath $ShopDbCredentialFile)) { return '' }
try {
Add-Type -AssemblyName System.Security -ErrorAction Stop
$protected = [System.IO.File]::ReadAllBytes($ShopDbCredentialFile)
$plain = [System.Security.Cryptography.ProtectedData]::Unprotect(
$protected, $null, [System.Security.Cryptography.DataProtectionScope]::LocalMachine)
$record = [System.Text.Encoding]::UTF8.GetString($plain) | ConvertFrom-Json
} catch {
Log "WARN could not read the endpoint credential ($ShopDbCredentialFile): $($_.Exception.Message)"
return ''
}
$credential = [string]$record.credential
if (-not $credential.StartsWith('sdbe_')) {
Log 'WARN endpoint credential file has no usable credential; ignoring it'
return ''
}
if (-not $TargetUrl.StartsWith('https://', [System.StringComparison]::OrdinalIgnoreCase)) {
Log "WARN not sending the endpoint credential over a non-https URL ($TargetUrl)"
return ''
}
# Never hand a credential to a server other than the one that issued it.
$issuer = ([string]$record.baseurl).Trim().TrimEnd('/')
if ($issuer -and -not ($TargetUrl.TrimEnd('/') + '/').StartsWith($issuer + '/', [System.StringComparison]::OrdinalIgnoreCase)) {
Log "WARN endpoint credential was issued by $issuer, not the target $TargetUrl; not sending it"
return ''
}
return $credential
}
function Get-ShopDbHttpFailure($ErrorRecord) {
# Status and error envelope code (data.error.code) from a failed request.
$status = 0
$code = ''
$text = ''
try { $status = [int]$ErrorRecord.Exception.Response.StatusCode } catch {}
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
$text = $ErrorRecord.ErrorDetails.Message
} else {
try {
$reader = New-Object System.IO.StreamReader($ErrorRecord.Exception.Response.GetResponseStream())
$text = $reader.ReadToEnd()
$reader.Dispose()
} catch {}
}
if ($text) {
try { $code = [string](($text | ConvertFrom-Json).data.error.code) } catch {}
}
$presented = ''
$matched = $false
if ('ShopDbServerPinValidator' -as [type]) {
$presented = [string][ShopDbServerPinValidator]::LastPresented
$matched = [ShopDbServerPinValidator]::IsPinned($presented)
}
# Innermost exception text: the outer WebException hides the real cause
# (unexpected EOF, connection reset, trust failure).
$message = ''
$exception = $ErrorRecord.Exception
while ($exception) {
if ($exception.Message) { $message = $exception.Message }
$exception = $exception.InnerException
}
return @{ Status = $status; Code = $code; PresentedPin = $presented; PinMatched = $matched; Message = $message }
}
function Write-ShopDbAuthFailure($Failure) {
switch ($Failure.Code) {
'CREDENTIAL_INVALID' {
Log ('ERROR 401 CREDENTIAL_INVALID: the server does not accept this PC''s endpoint credential ' +
'(revoked or unknown). The endpoint agent deletes it and re-enrolls on its own.')
}
'SHARED_KEY_REFUSED' {
Log ('ERROR 401 SHARED_KEY_REFUSED: the server refuses the shared key for this PC, because ' +
'the PC is enrolled or the shared key is switched off. Reports need the endpoint agent ' +
"credential: check agent 1.20+ is installed, running and enrolled ($ShopDbCredentialFile).")
}
'ASSET_MISMATCH' {
Log ('ERROR 403 ASSET_MISMATCH: this PC''s credential belongs to a different asset than the ' +
'hostname in the report. Check the hostname and the enrollment in ShopDB.')
}
default {
if ($Failure.Status -eq 0 -and $Failure.PresentedPin) {
# A pin was seen, but only an absent pin is a pin failure. A
# matched pin means the handshake passed and something later
# failed (server dropped mid-transfer, timeout).
if ($Failure.PinMatched) {
Log "ERROR request failed after the server key pin matched ($($Failure.PresentedPin)): $($Failure.Message)"
} else {
Log "ERROR TLS refused: server key pin $($Failure.PresentedPin) is not in ServerPins"
}
}
}
}
}
# Read a value from the 64-bit HKLM view explicitly. If this runs under 32-bit
# PowerShell the HKLM: PSDrive silently redirects to WOW6432Node, where the
# 64-bit agent's config is not, and the upgrade would look unconfigured.
function Get-Hklm64Value {
param([string] $SubKey, [string] $Name)
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey(
[Microsoft.Win32.RegistryHive]::LocalMachine,
[Microsoft.Win32.RegistryView]::Registry64)
try {
$key = $base.OpenSubKey($SubKey)
if (-not $key) { return $null }
try { return $key.GetValue($Name) } finally { $key.Dispose() }
}
finally { $base.Dispose() }
}
# Config resolution mirrors the agent's own order of preference, so the upgrade
# reaches the same server the agent is actually talking to. Registry first;
# appsettings.json is the fallback for locked-down hosts (CyberArk/EPM) that
# gate the HKLM hive against reads.
function Get-AgentConfig {
$baseUrl = Get-Hklm64Value -SubKey 'SOFTWARE\GE\ShopDB' -Name 'BaseUrl'
$apiKey = Get-Hklm64Value -SubKey 'SOFTWARE\GE\ShopDB' -Name 'CollectorKey'
$pins = @()
# Read even when the registry is complete: serverpins lives only here.
$settings = Join-Path $InstallDir 'appsettings.json'
if (Test-Path $settings) {
try {
$json = Get-Content $settings -Raw | ConvertFrom-Json
if ([string]::IsNullOrWhiteSpace($baseUrl) -or [string]::IsNullOrWhiteSpace($apiKey)) {
if ([string]::IsNullOrWhiteSpace($baseUrl)) { $baseUrl = $json.baseurl }
if ([string]::IsNullOrWhiteSpace($apiKey)) { $apiKey = $json.apikey }
Write-Log 'read config from appsettings.json (registry unavailable or incomplete)'
}
foreach ($value in @($json.serverpins)) {
if ($null -eq $value) { continue }
$pin = ([string]$value -replace '[\s:]', '').ToLowerInvariant()
if ($pin -match '^[0-9a-f]{64}$') { $pins += $pin }
}
}
catch {
Write-Log "could not parse appsettings.json: $($_.Exception.Message)" 'WARN'
}
}
if ([string]::IsNullOrWhiteSpace($baseUrl)) {
throw 'No BaseUrl found in the registry or appsettings.json. This endpoint does not look installed; run install-shopdbagent.ps1 instead.'
}
return @{ BaseUrl = $baseUrl.TrimEnd('/'); ApiKey = [string]$apiKey; ServerPins = $pins }
}
# ---------------------------------------------------------------------------
# First pass: relaunch detached and get out of the way. See header.
# ---------------------------------------------------------------------------
if (-not $Detached) {
New-Item -ItemType Directory -Force -Path $WorkDir, $LogDir | Out-Null
$selfCopy = Join-Path $WorkDir 'upgrade-shopdbagent.ps1'
Copy-Item -Path $PSCommandPath -Destination $selfCopy -Force
# Copied out of the agent's own temp dir on purpose: the delivered script
# may be cleaned up by the agent once the command returns, which would
# delete this file out from under the detached run.
Start-Process -FilePath 'powershell.exe' `
-ArgumentList '-NoProfile', '-ExecutionPolicy', 'Bypass', `
'-File', "`"$selfCopy`"", '-Detached' `
-WindowStyle Hidden | Out-Null
Write-Host 'Upgrade handed off to a detached process; this agent will restart shortly.'
exit 0
}
# ---------------------------------------------------------------------------
# Second pass (detached): the actual upgrade.
# ---------------------------------------------------------------------------
New-Item -ItemType Directory -Force -Path $WorkDir, $LogDir | Out-Null
Write-Log '== ShopDB Endpoint Agent self-upgrade =='
try {
$config = Get-AgentConfig
Write-Log "server: $($config.BaseUrl)"
# 1. Download BEFORE stopping anything. A failed download must leave the
# running agent untouched rather than stranding the endpoint with a
# stopped service and no replacement binaries.
$zipPath = Join-Path $WorkDir 'agent-package.zip'
if (Test-Path $zipPath) { Remove-Item $zipPath -Force }
$url = $config.BaseUrl + $PackagePath
Write-Log "downloading package from $url ..."
# TLS 1.2 explicitly: Windows PowerShell 5.1 still defaults to older
# protocols the server will refuse.
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
# Get-ShopDbServerPins already returns an array; wrapping it in @() made
# several pins one joined string. Registry wins when the value exists,
# even empty; appsettings pins apply only when it is absent.
$serverPins = Get-ShopDbServerPins
if (-not (Test-ShopDbServerPinsValue)) {
foreach ($pin in $config.ServerPins) {
if ($serverPins -notcontains $pin) { $serverPins += $pin }
}
}
if ($serverPins.Count -gt 0) {
if (-not (Enable-ShopDbServerPins $serverPins)) {
throw 'ServerPins is set but the pin check could not be enabled; not downloading.'
}
Write-Log "server pin check on ($($serverPins.Count) pin(s))"
}
$credential = Get-ShopDbEndpointCredential $config.BaseUrl
$attempts = @()
if ($credential) { $attempts += ,@('credential', @{ 'X-Endpoint-Credential' = $credential }) }
if ($config.ApiKey) { $attempts += ,@('shared key', @{ 'X-API-Key' = $config.ApiKey }) }
if ($attempts.Count -eq 0) {
throw 'No endpoint credential and no CollectorKey on this endpoint; cannot authenticate the download.'
}
# Credential first. A refused credential (revoked) falls back to the shared
# key once, which the server admits again for an un-enrolled PC.
$downloaded = $false
foreach ($attempt in $attempts) {
try {
Invoke-WebRequest -Uri $url -Headers $attempt[1] `
-OutFile $zipPath -UseBasicParsing -TimeoutSec 600
Write-Log "authenticated with $($attempt[0])"
$downloaded = $true
break
}
catch {
$failure = Get-ShopDbHttpFailure $_
Write-Log "download with $($attempt[0]) failed: $($_.Exception.Message)" 'WARN'
Write-ShopDbAuthFailure $failure
if ($failure.Code -ne 'CREDENTIAL_INVALID') { throw }
}
}
if (-not $downloaded) { throw 'Package download was refused.' }
$size = (Get-Item $zipPath).Length
Write-Log "downloaded $size bytes"
if ($size -lt 1MB) {
throw "Package is only $size bytes; refusing to install a payload this small."
}
# 2. Unpack to a staging dir and validate the payload BEFORE touching the
# install. Expanding straight over Program Files would leave a partially
# replaced install if the archive turned out to be short a binary.
$stage = Join-Path $WorkDir 'stage'
if (Test-Path $stage) { Remove-Item $stage -Recurse -Force }
New-Item -ItemType Directory -Force -Path $stage | Out-Null
Expand-Archive -Path $zipPath -DestinationPath $stage -Force
$newAgent = Join-Path $stage $AgentExeName
if (-not (Test-Path $newAgent)) {
throw "$AgentExeName not found in the downloaded package; aborting before any change."
}
$newTray = Join-Path $stage $TrayExeName
$haveTray = Test-Path $newTray
Write-Log ("payload validated (agent{0})" -f $(if ($haveTray) { ' + tray' } else { ', no tray' }))
# 3. Keep the current binaries so a failed swap can be undone. The config
# files are deliberately NOT part of this - they are never modified.
$backup = Join-Path $WorkDir 'backup'
if (Test-Path $backup) { Remove-Item $backup -Recurse -Force }
New-Item -ItemType Directory -Force -Path $backup | Out-Null
foreach ($name in @($AgentExeName, $TrayExeName)) {
$existing = Join-Path $InstallDir $name
if (Test-Path $existing) { Copy-Item $existing -Destination $backup -Force }
}
Write-Log 'backed up current binaries'
# 4. Stop the service and kill lingering processes. Until this completes
# the exes are locked and the copy would fail with a file-in-use error.
$svc = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($svc -and $svc.Status -ne 'Stopped') {
Write-Log "stopping service $ServiceName ..."
try { Stop-Service -Name $ServiceName -Force -ErrorAction Stop }
catch { Write-Log "could not stop service: $($_.Exception.Message)" 'WARN' }
}
$trayBase = [System.IO.Path]::GetFileNameWithoutExtension($TrayExeName)
foreach ($name in @('shopdbagent', $trayBase)) {
foreach ($p in (Get-Process -Name $name -ErrorAction SilentlyContinue)) {
Write-Log "killing $($p.ProcessName) (pid $($p.Id))"
try { Stop-Process -Id $p.Id -Force -ErrorAction Stop } catch { }
}
}
Start-Sleep -Milliseconds 750
# 5. Swap the binaries. On failure, restore the backup and restart, so a
# broken upgrade leaves a RUNNING old agent rather than a dead endpoint
# that can no longer be reached to fix remotely.
try {
Copy-Item -Path $newAgent -Destination $InstallDir -Force
if ($haveTray) { Copy-Item -Path $newTray -Destination $InstallDir -Force }
Write-Log 'binaries replaced'
}
catch {
Write-Log "copy failed: $($_.Exception.Message); rolling back" 'ERROR'
Get-ChildItem $backup -File | ForEach-Object {
Copy-Item $_.FullName -Destination $InstallDir -Force
}
Start-Service -Name $ServiceName -ErrorAction SilentlyContinue
throw
}
# 6. Clear Mark-of-the-Web. Files extracted from a downloaded archive carry
# a Zone.Identifier stream; SmartScreen blocks the unsigned exe on first
# run, which would leave the service unable to start.
Get-ChildItem -Path $InstallDir -File | ForEach-Object {
try { Unblock-File -Path $_.FullName -ErrorAction Stop } catch { }
}
# 7. Restart and verify it stays up. A service that starts and immediately
# faults is a failed upgrade, so confirm rather than assume.
Write-Log "starting service $ServiceName ..."
Start-Service -Name $ServiceName
Start-Sleep -Seconds 5
$svc = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if (-not $svc -or $svc.Status -ne 'Running') {
Write-Log 'service did not stay running; rolling back' 'ERROR'
Get-ChildItem $backup -File | ForEach-Object {
Copy-Item $_.FullName -Destination $InstallDir -Force
}
Start-Service -Name $ServiceName -ErrorAction SilentlyContinue
throw 'Upgraded service failed to start; rolled back to the previous build.'
}
$installed = (Get-Item (Join-Path $InstallDir $AgentExeName)).VersionInfo.FileVersion
Write-Log "upgrade complete; agent version now $installed"
# 8. Relaunch the tray for the interactive user.
#
# This CANNOT use the installer's `explorer.exe <path>` trick. That works in
# install-shopdbagent.ps1 only because the installer runs AS THE USER - note
# its own guard, `if ($interactiveUser -ne "SYSTEM")`. This script always
# runs as SYSTEM in session 0, where `explorer.exe <path>` hands the request
# to a shell that is not reachable from here and then exits 0. Start-Process
# would report success and we would log a relaunch that never happened,
# which is exactly the bug this replaces.
#
# It also must NOT descend into the session with a token-duplicating launch.
# UserContextSpool documents that WTSQueryUserToken is a privileged operation
# that endpoint protection blocks on these hardened hosts - the service tried
# that direction before and it produced nothing on a live endpoint.
#
# So the tray is started the same way its own data path is pinned: against
# the known ShopFloor account. Task Scheduler performs the session/token
# transition, and an Interactive principal with no RunLevel starts the tray
# at the user's normal integrity - preserving the de-elevation the installer
# wanted, because an elevated unsigned tray trips the CyberArk/EPM broker.
#
# Best-effort by design: the agent is already upgraded and running, and the
# HKLM Run key the installer wrote still starts the tray at next logon. A
# late tray is not worth failing - or rolling back - a good agent upgrade.
if ($haveTray) {
$trayPath = Join-Path $InstallDir $TrayExeName
# Same pinned account the spool uses (FileLoggerProvider.ShopFloorUserName):
# every shop-floor endpoint signs in as this local account, and the tray
# must run as it or the mapped-drive probe reports the wrong account's
# (empty) drive table. Only relaunch if that profile is actually present.
$trayUser = 'ShopFloor'
$trayUserProfile = Join-Path $env:SystemDrive (Join-Path 'Users' $trayUser)
if (Test-Path $trayUserProfile) {
$taskName = 'ShopDB tray upgrade relaunch'
try {
# Clear a leftover from an interrupted earlier upgrade so we do
# not stack duplicates.
if (Get-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue) {
Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue
}
$action = New-ScheduledTaskAction -Execute $trayPath
$principal = New-ScheduledTaskPrincipal -UserId $trayUser -LogonType Interactive
$settings = New-ScheduledTaskSettingsSet `
-AllowStartIfOnBatteries -DontStopIfGoingOnBatteries `
-ExecutionTimeLimit ([TimeSpan]::Zero)
Register-ScheduledTask -TaskName $taskName -Action $action `
-Principal $principal -Settings $settings -Force | Out-Null
Start-ScheduledTask -TaskName $taskName
# Verify rather than assume. The old code logged success off a
# Start-Process that could not fail, so a missing tray looked
# like a clean upgrade. Poll for the real process instead.
$trayBaseName = [System.IO.Path]::GetFileNameWithoutExtension($TrayExeName)
$running = $null
for ($i = 0; $i -lt 10; $i++) {
Start-Sleep -Milliseconds 500
$running = Get-Process -Name $trayBaseName -ErrorAction SilentlyContinue
if ($running) { break }
}
if ($running) { Write-Log "tray relaunched as $trayUser" }
else {
Write-Log ('tray did not appear after relaunch; it will start at next ' +
'logon via the HKLM Run key') 'WARN'
}
# One-shot hand-off, not a persistent artifact: leaving it would
# surface in the uninstaller's leftover sweep and in task audits.
Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue
}
catch {
Write-Log ("could not relaunch tray: $($_.Exception.Message); " +
'it will start at next logon') 'WARN'
}
}
else {
Write-Log ("no $trayUser profile at $trayUserProfile; " +
'leaving the tray to start at next logon') 'WARN'
}
}
Remove-Item $stage -Recurse -Force -ErrorAction SilentlyContinue
Remove-Item $zipPath -Force -ErrorAction SilentlyContinue
exit 0
}
catch {
Write-Log "UPGRADE FAILED: $($_.Exception.Message)" 'ERROR'
exit 1
}