- Settings > Endpoints > Enrollment. Migration tab: readiness counts and
progress, each PC's state with the reasons it is blocked, "Upgrade
selected" and "Upgrade next 25", and live wave progress. Credentials tab:
enrolled PCs with revoke, pending requests with approve and reject, and the
shared key and auto-enroll switches. Turning the shared key off asks for
confirmation and shows how many PCs are not enrolled.
- The server certificate page shows the SPKI pin to put in the GE-Enforce
pin entry.
- The agent upgrade script and the asset and printer reporters send the PC's
credential when it has one (over https, only to the issuing server), fall
back to the shared key, check server pins, and log refusals clearly.
- New Set-ShopDBServerPins.ps1 (GE-Enforce entry) and Get-ShopDBServerPin.ps1.
- AGENT-CONTRACT.md documents per-PC credentials; the rollout runbook covers
phases, gates, rollback, pin and wave entries, and every place the shared
key lives and how to clear it at cutover.
- Removed the 09-14 certificate design notes and dated session reports, and
two VM notes that carried a live collector key and advised plain http.
- Corrected the GE-Enforce collector note, which claimed a plain registry
token is protected by ACL.
Agent 1.20.0, tray 1.7.0. Ship together.
Agent:
- Every HTTP client, including script and package downloads, is built by one
AgentHttpFactory. Server pins (SPKI SHA-256, current and next) come from
HKLM ServerPins or appsettings serverpins; with a pin set only a matching
key is trusted, otherwise the normal chain check applies.
- EnrollmentService enrolls once using the shared key the PC already has,
stores the credential DPAPI-protected in a SYSTEM-only directory, and swaps
to it without a restart. Pending requests retry every 15 minutes; an old
server without the enroll route leaves the agent on the shared key. A
revoked credential is deleted and the PC re-enrolls.
- A credential is only ever sent to the server that issued it.
- Telemetry posts to /api/endpoints/collect when using a credential.
- A named pipe lets the interactive user ask the agent to send status, send a
report, list alerts and acknowledge one. The report sidecar and screenshot
must sit in the caller's own profile and not be a link.
- --once and --report run as a user become pipe clients.
- appsettings.json is restricted to SYSTEM and Administrators once the pipe is
up.
- Removed client certificate renewal and expiry monitoring.
Tray:
- Holds no key and reads no agent configuration. Alerts, acknowledgements,
Send Status and Report an issue all go through the agent's pipe.
- An acknowledgement is remembered only when the agent confirms it.
- When the agent is not running, the menu says so and alert polling retries
quietly.
Tray 1.6.11.
- One alert with a null or malformed field no longer throws away the whole
alert list; nullable server fields are tolerated.
- Keystrokes can no longer acknowledge an alert the operator has not seen.
There is no default button, focus starts on the message, and acknowledging
takes a mouse click after the alert has been on screen for 1.5 seconds.
- Long messages scroll instead of being cut off, and an ampersand is shown as
typed.
- A timeout closes the alert without acknowledging it.
- Missing type names, titles and colours fall back to safe defaults.
- Shutdown waits on cancellation instead of sleeping, and closes an open alert
before the client is disposed.
- The acknowledged-alerts store moved from ProgramData, which the operator
cannot write, to the user's local app data, guarded by a mutex. Acked alerts
used to reappear at every logon.
Agent 1.19.0.
- With no pinned thumbprint the collector, command and fleet alert clients
accepted any server certificate. They now require a valid chain again.
Production uses a self-signed certificate, so this build must not reach bays
before server pinning ships with it.
- SYSTEM script runs read stdout and stderr while the script runs, cap what is
kept, and kill the whole process tree after five minutes. A chatty script
used to fill the pipe and hang the command channel until a service restart.
- Scripts and update_app packages are staged in a directory only SYSTEM and
Administrators can touch, under a random name, with the hash checked in
memory and again on disk. A standard user could previously swap the file
between the hash check and execution.
- The user-context transcript lives in a SYSTEM-created directory, and SYSTEM
refuses to follow a link when reading or deleting it.
- Strict mode is inserted after any param block, #Requires or using lines,
keeps line numbers and writes UTF-8 with a BOM.
- A user-context run that reports no exit code now fails.
- collect_log refuses setup answer files, appsettings files, the agent's own
directories, key files and links.
- The startup renewal check is capped at 15 seconds and exits non-zero when it
wants a restart, so service recovery restarts the agent.
This commit adds:
- Database migrations for app_delta fingerprints and client certificates
- Scripts for certificate generation and fingerprint backfilling
- Comprehensive security audit reports and findings tracking
- Implementation documentation for certificate authentication
- Test utilities for certificate renewal and fingerprinting
- GE-ENFORCE collector fix documentation
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Build Fix: .NET 8 API changes
Problem:
- Agent failed to compile with .NET 8.0.424
- Error: 'File' does not contain a definition for 'GetAccessControl'
- Error: 'File' does not contain a definition for 'SetAccessControl'
Root Cause:
- File.GetAccessControl() and File.SetAccessControl() removed in .NET 5+
- Moved to System.IO.FileSystem.AccessControl package as extension methods
- Methods now on FileInfo/DirectoryInfo classes, not File static class
Solution:
1. Added package reference:
- System.IO.FileSystem.AccessControl v5.0.0 (ShopDbEndpointAgent.Core.csproj)
2. Added using directive:
- using System.Security.AccessControl; (CertificateRenewal.cs)
3. Changed API usage (CertificateRenewal.cs:290-298):
Before:
var acl = File.GetAccessControl(keyFile);
File.SetAccessControl(keyFile, acl);
After:
var fileInfo = new FileInfo(keyFile);
var acl = fileInfo.GetAccessControl();
fileInfo.SetAccessControl(acl);
Build Results:
- ShopDbEndpointAgent.Core: SUCCESS (1 warning, 0 errors)
- ShopDbEndpointAgent: SUCCESS (0 warnings, 0 errors)
- ShopDbEndpointAgent.Tray: SUCCESS (0 warnings, 0 errors)
- Published to agent/publish/agent/ and agent/publish/tray/
Context:
- Private key ACL grants required for tray app access (CertificateRenewal.cs:290)
- BUILTIN\Users group needs Read+ReadAndExecute on private key file
- Enables certificate renewal from both service and tray contexts
Related Changes:
- CertificateExpiryMonitor.cs (periodic monitoring, commit bb301c6)
- CertificateRenewal.cs (server cert validation, commit 702eeb3)
- All security fixes operational after this build
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
CRITICAL OPERATIONAL FIX: Agents running 90+ days without restart
Problem:
- Certificate renewal check only ran at service startup
- Agents running continuously for 30+ days never renewed certificates
- Certificates would expire at day 730 causing authentication failures
- User confirmed agents routinely run 90+ days without restart
Solution:
- Added CertificateExpiryMonitor background service
- Checks certificate expiry every 24 hours during runtime
- Triggers renewal when < 30 days remaining
- Gracefully stops service (Windows Service Manager restarts it)
- New process picks up renewed certificate
Implementation:
- CertificateExpiryMonitor.cs: Background service (IHostedService)
- Registered in Program.cs as hosted service
- First check at 1 hour after startup, then every 24 hours
- Uses same CertificateRenewal logic as startup check
- Calls IHostApplicationLifetime.StopApplication() after renewal
Timeline Example:
- Day 0: Agent starts with 730-day cert
- Day 1-89: Agent runs continuously (no restart)
- Day 90: Periodic check runs, cert has 640 days left, continue
- Day 700: Periodic check runs, cert has 30 days left, renews
- Agent stops gracefully, service manager restarts
- Day 701: Agent continues with new 730-day cert
Without this fix:
- Agent starts day 0, runs continuously
- Day 730: Cert expires, no renewal check ever ran
- Authentication fails at TLS handshake
- Agent dead until manual intervention
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Addresses all critical and high-severity findings from Fable security review.
CRITICAL FIX#1: Server Certificate Validation Bypass
- File: CertificateRenewal.cs
- Issue: Agent accepted ANY server certificate when ServerCertThumbprint not configured
- Fix: Now validates against Windows trusted roots when pinning not configured
- Impact: Prevents MITM attacks during renewal requests
CRITICAL FIX#2: Certificate Revocation Check Incomplete
- Files: asset.py, client_cert.py, migration 0022
- Issue: Revocation check used hasattr() and silently skipped when columns missing
- Fix: Applied migration 0022, uncommented revocation columns, removed hasattr()
- Impact: Revoked certificates can no longer authenticate or renew
CRITICAL FIX#3: Expired Certificates Cannot Renew
- Files: routes.py (renewal endpoint), CertificateRenewal.cs
- Issue: Expired certs fail at TLS handshake before reaching renewal endpoint
- Fix: Added API key + hostname fallback for renewal when cert expired
- Agent now sends hostname in request body for API key authentication
- Impact: Agents can renew expired certificates, 30-day window becomes effective
HIGH FIX#4: Hostname Injection Vulnerability
- File: routes.py
- Issue: No validation before passing hostname to certificate generation
- Fix: Added regex validation - 1-64 chars, alphanumeric plus dot/hyphen/underscore
- Impact: Prevents injection attacks via malicious hostnames
HIGH FIX#5: Rate Limit Bypass via Clock Manipulation
- File: routes.py
- Issue: Naive UTC datetime arithmetic vulnerable to timezone changes
- Fix: Use timezone-aware datetime with explicit UTC conversion
- Impact: Rate limit cannot be bypassed via clock manipulation
HIGH FIX#6: Hard Process Termination Loses In-Flight Work
- File: Program.cs
- Issue: Environment.Exit(0) aborted in-flight work and could corrupt state
- Fix: Graceful return from RunHostAsync() instead of Environment.Exit()
- Impact: Service exits cleanly, no lost telemetry or corrupted local state
HIGH FIX#7: Missing Database Transaction Isolation
- File: routes.py
- Issue: Race condition in concurrent renewal requests
- Fix: Added with_for_update() row-level locking for both auth paths
- Impact: Prevents concurrent renewals from bypassing rate limit
MEDIUM FIX#8: Missing Audit Logging for Failures
- File: routes.py
- Issue: Failed renewal attempts not captured in audit trail
- Fix: Added AuditLog.log() for cert generation and database update failures
- Impact: Security monitoring can detect repeated renewal failures
SECURITY HARDENING:
- Server cert validation: Never accepts certificates blindly
- Revocation enforcement: Fail closed if columns missing
- Expired cert handling: Dual-auth allows renewal while maintaining security
- Input validation: Hostname must match safe character set
- Transaction safety: Row locks prevent race conditions
- Audit trail: All failures logged for security monitoring
OPERATIONAL IMPROVEMENTS:
- Agents can now renew expired certificates (no more manual intervention)
- Graceful shutdown prevents data loss during renewal
- Rate limiting immune to timezone issues
- Comprehensive audit trail for troubleshooting
ADR COMPLIANCE:
- Maintains ADR-001 (Asset as platform contract)
- Maintains ADR-006 (Collector contract - backward compatible)
- Database naming (snake_case columns)
- No unicode/emojis
- Security best practices: ALL CRITICAL/HIGH issues resolved
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
AGENT SELF-RENEWAL:
- Agent checks certificate expiry on startup
- If < 30 days remaining, requests renewal from server
- Server generates new certificate and updates database
- Agent installs new cert, updates config, and restarts
SERVER-SIDE (/api/endpoints/certificate/renew):
- Requires authentication via existing (expiring) client certificate
- Rate limited: 1 renewal per asset per 24 hours
- Generates 2-year self-signed certificate with cryptography library
- Computes SHA1 fingerprint (matches nginx $ssl_client_fingerprint)
- Updates assets table with new fingerprint and expiration
- Returns PFX/P12 certificate with random password to agent
- Audit logs all renewal operations
AGENT-SIDE (CertificateRenewal.cs):
- CheckAndRenewIfNeeded() called on service startup before host starts
- Finds current certificate by thumbprint in LocalMachine\My store
- Checks expiry: if < 30 days, POST to /api/endpoints/certificate/renew
- Authenticates renewal request with expiring certificate (mTLS)
- Receives PFX + password in JSON response
- Imports new certificate to LocalMachine\My store
- Grants BUILTIN\Users read access to private key (for tray app)
- Updates appsettings.json with new thumbprint
- Exits with code 0 so service manager restarts agent with new cert
CERTIFICATE GENERATION (certificate_renewal.py):
- Uses cryptography library to generate RSA 2048-bit key pairs
- Self-signed certificates with CN=hostname, 2-year validity
- Extended key usage: CLIENT_AUTH
- Subject Alternative Name: hostname as DNS name
- Exports as password-protected PFX/P12 for Windows import
- SHA1 fingerprint computed from DER encoding (nginx compatible)
STARTUP FLOW:
1. Agent service starts (Program.cs RunHostAsync)
2. CertificateRenewal.CheckAndRenewIfNeeded() runs BEFORE host starts
3. If cert expires in < 30 days, requests renewal from server
4. If renewal succeeds, agent exits (Environment.Exit(0))
5. Windows Service Manager automatically restarts the service
6. New process picks up updated thumbprint from appsettings.json
7. Agent continues with renewed certificate
SECURITY:
- Renewal requires valid (not expired) existing certificate
- Rate limit prevents renewal loop abuse
- Server validates cert not revoked before renewal
- Audit logging tracks all renewal operations
- PFX password is random, used once during import, never stored
PERFORMANCE:
- Renewal check adds ~100-200ms to service startup
- Only triggers when expiry < 30 days (rare operation)
- No impact on steady-state agent performance
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
CRITICAL FIXES:
- Enforce cert authentication on collector telemetry endpoint
- Add authorization check in report_command_result (prefer cert over hostname)
- Implement server certificate pinning in C# agents (optional)
- Add constant-time fingerprint comparison to prevent timing attacks
- Validate fingerprint format (40 or 64 hex chars only)
AUTHENTICATION:
- Created shopdb/core/auth/client_cert.py with authenticate_by_client_cert()
- Extracts fingerprint from nginx header (SHA1), IIS environ, or ARR header
- Validates against assets.client_cert_fingerprint in database
- Checks certificate expiration (cert_expires_at column)
- Falls back to API key when cert not present (dual-auth during rollout)
AUTHORIZATION:
- shopdb/core/api/collector.py: Added cert auth to generic_collect() before
API key check. Sets g.cert_authenticated_asset_id when cert validates.
- plugins/endpoints/plugin.py: apply_collector_payload() compares
g.cert_authenticated_asset_id with payload's resolved assetid. Raises
PermissionError if mismatch (prevents agent A from submitting data for
asset B by changing hostname field).
- plugins/endpoints/api/routes.py: report_command_result() prefers
g.cert_authenticated_asset_id over hostname parameter. Requires either
cert or hostname for authorization.
AGENT CHANGES:
- AgentConfig.cs: Added ServerCertThumbprint config for certificate pinning
- CollectorClient.cs: Validates server cert thumbprint when configured
- CommandClient.cs: Validates server cert on WebSocket TLS
- FleetAlertClient.cs: Validates server cert on WebSocket TLS
- When ServerCertThumbprint is empty, accepts all certs (rollout compat)
REVOCATION SUPPORT:
- Added cert_revoked_at and cert_revocation_reason columns to assets table
- Migration 0022_client_cert_revocation.py adds columns
- Revocation check uses hasattr() for backward compatibility
- Revocation requires manual database UPDATE (instant kill switch)
SECURITY HARDENING:
- Constant-time fingerprint comparison via hmac.compare_digest()
- Format validation: must be 40 (SHA1) or 64 (SHA256) hex chars
- SHA1 documented as nginx limitation, acceptable for air-gapped self-signed
- Audit logging for failed authentication attempts (AuditLog.log)
DEPLOYMENT:
- Dual-auth during rollout: cert OR API key both work
- Agents without certs continue using API key (backward compatible)
- No impact on existing fleet until certs provisioned
- Rollout: server first, pilot certs, fleet rollout, retire API key
Addresses Fable security review findings: 1 critical + 5 high vulnerabilities
related to authentication bypass, authorization flaws, and timing attacks.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changed FleetAlertStore to save acknowledgements in:
ProgramData/ShopDB/FleetAlerts/acknowledged-alerts.json
Instead of:
ProgramData/ShopDbAgent/acknowledged-alerts.json
Now matches the standard folder structure used by other agent components.
Log Collector: a new collect_log action reads a plaintext log file from a
known path and returns its tail for viewing and download. Rides the existing
endpoints.updateapp grant rather than inventing one, since collecting a log is
the same trust level as pushing an application. Collected logs land in their
own table with per-asset retention of 20 instead of overloading the command
audit row, and the agent refuses UNC, traversal, wildcards, directories and
non-text content independently of the server check.
collect_log is POLICY_EXEMPT. Every existing Configuration Policy predates the
action and so cannot name it, and the intersection in ALLOWED_ACTIONS_FOR drops
anything unnamed - so every policied endpoint refused log collection outright,
including for admins, who saw a permission-shaped error for a grant they held.
Exemption from policy is not exemption from permission: the elevated grant
still applies, and a regression test asserts both halves so a future edit
cannot quietly loosen the grant while "fixing" the policy gate.
Script output capture: successful runs can return output, gated per script by
captureoutput. Failures always returned it; success never did, which left a
green badge as the only evidence a run produced.
Timestamps: Activity and Operator Reports rendered date-only because the shared
formatter's 'date' branch drops the clock time. Run Scripts additionally
labels its timestamp, since a bare time meant "completed" on a terminal row but
"queued" on an in-flight one - indistinguishable, and the wrong reading.
Agent 1.16.0. Minor: collect_log is a new action name, so an older agent asked
for it fails closed on the unknown-action default.
Brings in wilm commits:
- Render script surfaces correctly in dark mode
- Stop scripts reporting success after doing nothing
- Correct two gate commands that pass without running
- Gate the script library on the grant that pushes scripts
- Let the policy list edit the rows the API accepts
- Cap plugin revision ids at the 32 chars MySQL stores
- Split shipped monitor bundle into per-monitor policies
Resolved conflict in agent ShopDbEndpointAgent.csproj by merging:
- wilm 1.14.0 script verdict and output capture features
- wj 1.13.1-1.13.5 tray coordination releases
- Bumped to 1.14.1 for coordinated agent+tray release
Brings in the fleet alert notification concept, the operator report email
alerts, and the tray work that goes with them (tray 1.6.5).
Two files needed resolution:
- ShopDbEndpointAgent.csproj: wj carried 1.13.1 through 1.13.4, all
tray-coordination bumps with no agent code behind them. Ours carried
1.14.0, which is real code. Kept every changelog entry and took 1.14.0
as the version, since it already supersedes the 1.13.x line.
- plugins/endpoints/api/routes.py auto-merged; verified the errormode
descriptor survived intact.
KNOWN RED, INHERITED NOT CAUSED: five backend tests fail on this merge -
authz on endpoints.acknowledge_fleet_alert, the public GET surface scan,
two plugin-contract import checks, and the notifications plugin
migration/model agreement. All five were confirmed failing on a clean
worktree of origin/wj before merging, so they arrive with the branch
rather than from the merge. They reflect the fleet alert work being
half-landed: the push path was a delivery proof of concept and the
server-side surface was still being stood up.
These are not fixed here on purpose. The next change replaces that
delivery path with the endpoint agent, so the notification persists on
the endpoint until the operator acknowledges it or the server closes it,
and the affected surfaces get their gating and migration settled as part
of that work rather than being patched twice.
A run_script command was judged solely on PowerShell's exit code, and
PowerShell treats most errors as non-terminating: it writes the error,
runs the next line, and still exits zero. The account-provisioning
script reported succeeded having provisioned nothing, and because no
output was captured there was no evidence to contradict the verdict.
Scripts now carry an errormode. Strict prepends $ErrorActionPreference
= 'Stop' so the first error ends the run and the exit code means what
it says. It is opt-in per script rather than global because bodies
authored against the permissive semantics rely on them - tolerating a
failing cleanup, probing with a cmdlet that errors when a thing is
absent, looping past a bad item - and would start aborting midway,
leaving endpoints half-configured. The default is unchanged, so every
existing script keeps running exactly as before.
The prologue is applied after digest verification, so the SHA-256 still
validates the bytes the server delivered.
stdout/stderr is captured on both paths: redirected streams for SYSTEM,
a transcript file under the ShopFloor profile for user-session runs,
since CreateProcessAsUser cannot inherit the service's pipes across the
session boundary. Output is always logged locally and returned to the
server only on failure, so routine output stays on the machine that
produced it instead of filling command history.
Also adds the optional per-script working directory, which the same
surfaces needed, and a collapsible explainer on the scripts page - the
dropdown is not self-explanatory, and an operator who does not know
about non-terminating errors has no reason to ever leave the default.
Agent 1.14.0. An older agent ignoring errormode keeps today's behaviour.
- Change device URL from API endpoint (/api/assets/{id}) to frontend page (/shopdb/pcs/{id})
- Add asset number to email subject line: "Operator Issue Report: {name} ({number})"
- Bump agent to 1.13.5 and tray to 1.6.6 for coordinated release
- Dispose GraphicsPath after creating Region to prevent GDI handle leak
- Track and dispose all Font objects to prevent GDI+ resource leak
- Join worker thread before disposal to ensure client not used after disposal
- Fix Screen.PrimaryScreen null dereference with fallback rectangle
- Propagate cancellation token to FetchAlertsAsync to stop orphaned requests
- Bump tray to 1.6.5, agent to 1.13.4
- Acknowledgment now marks locally first, then posts to server in background
- Status check timer runs async properly without blocking UI thread
- Both changes use Task.Run to avoid .GetAwaiter().GetResult() deadlocks
- Bump tray to 1.6.4, agent to 1.13.3
- Fleet alert forms now check every 30 seconds if notification is still active
- Automatically close if notification has ended or been deactivated
- Bump tray to 1.6.3, agent to 1.13.2
Style check fixes:
- Replace unicode box-drawing characters with ASCII (|-- instead of ├──)
- Replace emoji status markers with [OK]/[FAIL]/[WARN] tags
- Replace arrows (→ with ->, ↔ with <->) and em-dashes (— with --)
- Add venv-quick to excluded directories in check script
- Exclude darkwing report files (findings.sarif, report.md)
Backup finalization fix:
- Fix finalize-restore endpoint to not claim the upgrade sentinel
- Endpoint now checks for pending upgrade and returns {queued: true}
- Leaves UPGRADE_SENTINEL for background watcher to claim and process
- Prevents inline Alembic runs that can outlast request timeout
Claude Code context:
- Add .claude/ directory with INDEX.md and QUICK-REFERENCE.md
- Add CLAUDE.md with project overview and conventions
- Helps Claude Code understand project structure and rules
Fixes failing test: test_the_endpoint_does_not_run_the_upgrade_itself
Fixes naming/style CI check violations
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
VM cannot connect via HTTPS due to self-signed certificate errors.
Added:
- test-vm-connectivity.ps1: 5-step diagnostic for VM
- agent/QUICK-FIX-VM.md: Use HTTP for Hyper-V local network
- open-firewall-for-vm.ps1: Open Windows Firewall
Since VM and host are on same physical machine (Hyper-V internal
network), HTTP is acceptable for local monitoring.
For production: Use proper TLS certificates.
Configuration for deploying fleet agent on VM that reports to host PC:
- Host IP: 172.31.96.1
- Collector API key included
- Step-by-step build and install instructions
- Network and SSL troubleshooting
Completes the Docker deployment with endpoint monitoring capability.
Both agent loops made their first request the instant the service
started, and the service is installed start=auto - so a patch reboot put
the whole fleet on the wire together, twice per machine (a telemetry
report and a command poll). The existing jitterseconds does not help: it
spreads the gap between runs, not first contact.
Measured on a 700-endpoint burst: simultaneous arrival drove TLS
handshakes to 1482ms worst case and pinned CPU at 305% while the
database sat idle at 3 active queries. Spreading the same arrivals over
30s cut worst-case handshake to 63.7ms with no server-side change. The
server was never the constraint; simultaneity was. Raising the
SQLAlchemy pool was tried and rejected - it moved the queue from
SQLAlchemy into MySQL without adding throughput.
StartupJitter is shared by both workers so the clamp and the
randomisation cannot drift apart between them. Randomised in
milliseconds rather than whole seconds because a 30s window at second
granularity re-clumps a large fleet into 30 buckets of simultaneous
arrivals - the same pile-up in miniature. Clamped to 300s and warned
about when out of range: the command channel is how an operator would
correct a bad value, so an unbounded delay would remove the means of
recovery.
The cost is up to startupjitterseconds of extra latency before a
rebooted endpoint picks up a queued command, on top of a reboot the
machine was already unreachable for. The once-only console switch
bypasses the delay entirely, so smoke tests stay instant.
Agent 1.12.0. Minor because the appsettings schema gains a key; packages
without it fall back to the AgentConfig default. The shipped installer
template in siteagent.py carries the new key, so the containers need
rolling before newly built packages pick it up.
Corporate Tools tab on PC Details, gated by endpoints.deploy and liveness.
Starts already-registered scheduled tasks; the agent's ScheduledTaskPolicy
allowlist remains the authority on what may actually run.
Dell Command | Update split into two actions (Scan, Apply Updates) because
they have very different consequences. Apply Updates carries a persistent
breakglass warning in the card naming Tanium as the primary update utility,
rendered between the row title and description so it is visible at all times
rather than only in the confirm dialog. Destructive actions keep the typed
RUN confirmation.
Fix scheduledtask_run being stripped by Configuration Policy: the
intersection in ALLOWED_ACTIONS_FOR dropped it on any endpoint whose
effective policy was non-empty, so every Corporate Tools click failed with
"not permitted by policy" regardless of the caller's role. Corporate tools
are role-gated only, via POLICY_EXEMPT_ACTIONS; policy keeps authority over
monitors and service/process primitives.
Fix blank PC Details page: watch(detailtabs) evaluates its source during
setup and read canDeploy before its const declaration, throwing a TDZ
ReferenceError that aborted setup with no API request made.
Mapped-drive refresh moves to the tray. SYSTEM cannot probe user-scoped
drives, so the service requests a re-probe over a named event and waits for
the spool collectedat stamp to advance before collecting. The probe task is
registered as a ShopFloor user principal; the Users-group principal remains
only as a degraded fallback since a group member cannot start it.
Mapped drives reported Unavailable on endpoints where the operator could
see them fine. The data path was intact; the collection was running under
the wrong token. Mapped drives are per logon token, so an elevated probe
enumerates a different drive table than the interactive user.
Collection:
- Pin the user-context spool path so the probe and service agree on one
location, and persist/overwrite it instead of deleting on read.
- Refuse to collect while elevated (exit code 2) rather than emitting a
confident wrong answer, and stamp elevated/username on the spool.
- Reject spools stamped elevated or from an unexpected account on read,
so an admin run cannot masquerade as valid.
- Log netuse/smbmapping/userlistener counts so each run leaves evidence
of what it actually saw.
Install and launch context:
- Move collection to a 'ShopDB user-context probe' scheduled task under
S-1-5-32-545 at RunLevel Limited, the only mechanism that reliably runs
in the operator's own session. Verify registration and record it in the
install ledger.
- Stop starting the tray as a child of the elevated installer; explorer.exe
de-elevates without changing ACCOUNT, so an admin install produced an
admin tray. The Run key only fires at logon, which left the tray dead on
a machine already signed in, so add a transient 'ShopDB tray first launch'
task that starts it once in the existing session and unregisters itself.
- Hide the console window during --collect-usercontext. The agent is a
console exe; the tray used to mask this with CreateNoWindow, but the
scheduled task invokes it directly and flashed a window every cycle.
Installer edits are in _install_script() in plugins/endpoints/api/siteagent.py,
which generates the install.ps1 that actually ships. The standalone
agent/installer/install-shopdbagent.ps1 was never shipped once the package
moved to the UI builder and is deleted here to stop the divergence that
caused an earlier fix to land in a file no endpoint ever ran.
Versions: agent 1.10.1, tray 1.3.0. These share the pinned spool path and
must be deployed together.
Pre-existing test failures unrelated to this work (8, identical on a clean
tree): docs publishable/versions, plugin contract symlink scan, import
guard tamper check, security headers upload sandbox, and three endpoint
timestamp tests.
Mapped drives stopped reporting because the user-context path failed silently
in three separate places, and none of them left a trace to read back.
- Probe: launch via -EncodedCommand instead of staging a temp .ps1 and running
powershell.exe -File. A missing or quarantined script file fails BEFORE any
statement runs, so the script's own try/catch never sees it and the only
symptom is exit -196608.
- Spool: pin to the ShopFloor profile. Resolving it through WTSQueryUserToken
is privileged, endpoint protection blocks it, and on failure the tray
(writer) and service (reader) silently disagree about the path.
- Spool: consume after a successful parse, so a stopped tray cannot keep
re-reporting stale mapped drives as current. Unparseable files are left in
place for the next cycle.
- Logging: user-session components cannot write to Program Files or to the
SYSTEM-owned ProgramData log directory, and the file sink degrades to a
silent no-op when it cannot create its directory. Tray and probe now log to
C:\Users\ShopFloor\ShopDB\Logs, installs transcript to ProgramData, and the
sink records a SinkProblem instead of swallowing UnauthorizedAccessException.
- Installer: provision the operator spool/log directories with ShopFloor:Modify
before the tray first runs, and stop the transcript via trap so a throw
cannot orphan exactly the failed install worth reading.
- Upload cap 64 -> 128 MB: the self-contained tray publishes at ~69 MB and was
rejected on SIZE, but the size check returns before the PE is parsed, so the
operator only ever saw a version-detection failure.
AGENT-CONTRACT.md gains a normative section defining the three security
contexts, their log destinations, the fixed spool path and consume-on-read.
Agent 1.8.0 -> 1.8.1, tray 1.1.0 -> 1.1.1 so the deployed build is
identifiable from a log line.
Link speed and duplex are paired onto the existing interface entries by MAC from MSFT_NetAdapter, which is a second query because the IPEnabled config class carries no link state. Both fields stay null when unreadable: a disconnected adapter returns empty values on the live box, so defaulting would report every unplugged NIC as half-duplex.
Boot info keeps two facts separate rather than collapsing them. The registry says whether fast startup is configured; Kernel-Boot event 27 says what the last boot actually did. The registry cannot answer the second and the event can age out of the log, so neither substitutes for the other. This exists because uptime lies under fast startup.
Defender is collected and persisted but not surfaced, as specified: known false positives here would train operators to ignore the card. Ages are in days and a real zero means updated today, so the collector sends null when Defender did not report.
ATA SMART alone is not enough. MSStorageDriver_FailurePredictStatus returns zero instances on an NVMe machine, verified on a two-NVMe box where both drives were present and healthy, so an agent collecting only that class would report an empty disk health list on every modern endpoint and read as no problems.
MSFT_PhysicalDisk is the only NVMe-aware source; Win32_DiskDrive.Status covers ATA/SATA and hosts where the Storage namespace is unavailable. Both are collected and neither is reconciled away, so a disagreement between them stays visible.
MSFT_PhysicalDisk returns raw enum integers rather than the friendly strings Get-PhysicalDisk prints, so the mapping is done here to keep the wire self-describing. predictfailure is precomputed so an alert is a boolean test instead of a string match against a Windows enum; null means the drive did not answer, which is unknown rather than healthy.
Windows exposes mapped drives twice and the two views disagree. Captured on one
box at one moment, same two mappings: Get-SmbMapping reported OK for both while
net use reported Disconnected and Reconnecting. Collecting only Get-SmbMapping -
the nicer, more structured one - would have reported a healthy Q: on a machine
whose Q: was down, which is the exact failure this telemetry exists to catch.
So collect both and keep them apart. The probe emits netuse and smbmapping as
separate keys and does not reconcile them; the disagreement is the signal and
normalising it in the agent would discard it before anyone could see it.
The server merges by drive letter and shows the WORSE of the two. A drive that
either source calls broken is broken for the operator, and a drive neither
source gave a status for renders as Unknown rather than blank - an unannotated
row reads as healthy, so a failed check would present as a working drive.
Drives seen by only one source are kept, not dropped: Get-SmbMapping is absent
on some builds and session types, and requiring agreement would silently hide
every mapped drive on those machines.
No active reachability testing. Windows already knows the status, so touching
each share would be wasted work and would reintroduce the hang risk the bounded
probe exists to avoid.
The probe stays unelevated in the operator's session. UAC token separation
gives an elevated context a different mapped-drive table, often empty, so
elevating this later would quietly empty the card.
Adds a commented-out remap_drive seam in the command catalog. Not implemented,
but placed so the next person finds the user-session and elevation constraints
before writing it rather than after.
The merge logic has no test coverage yet; the docstring names the three
behaviours that must not be broken.
The service ran as SYSTEM and tried to launch the probe into the interactive
user's session. That path is fragile under CyberArk and produced no spool file
at all on a live endpoint: session-scoped data has to be gathered by something
already in the session, not reached down into from outside it.
Invert it. The tray already runs unelevated as the operator, so it invokes
shopdbagent.exe with the user-context verb on a timer and the service only
reads the resulting spool file. Nothing descends between security contexts.
That makes the tray the writer and the service an independent reader, so the
spool can no longer be consumed on read - deleting it would race the collector
and drop data the tray had just gathered. It is now read non-destructively and
aged out by its own collectedat value instead, which also means freshness is
decided by when the data was collected rather than by when it happened to be
read. The 60-minute read throttle goes with it; it delayed data that was
already sitting on disk.
Agent 1.8.0, tray 1.1.0. Both ship together - the tray invokes a verb this
agent build is the first to understand.
Curated events were re-read from the newest 25 entries per channel on every
collect, and the server appended whatever arrived. One real event was
therefore stored again on every cadence tick for as long as it stayed in
that window, so event counts measured how long a box had been up rather
than what had happened to it.
The agent now keeps a per-channel watermark of the last Windows event
record id it reported, persisted under ProgramData so a service restart
does not replay history, and sends eventrecordid with each event. The
server dedupes on (assetid, eventchannel, eventrecordid) behind a unique
index, so a replay from an older agent cannot double-store either.
Six inventory keys the agent had been sending all along - bios, disks,
domain, memory, osinfo, processor - were parsed, logged as unpersisted,
and dropped. They now have columns and are written on check-in, along
with user context, pending reboot, and the cpupercent/memorypercent/
idleseconds fields that were declared in the contract but always null.
Session-scoped data cannot be read from the agent's LocalSystem session 0
at all: mapped drives and user-owned listeners do not exist there. A probe
runs in the interactive user's session and hands results back through a
spool file. It is not yet producing output on a live endpoint; the column
and panel are in place and read empty until that is fixed.
Also bounds the user-context script wait. It was WaitForSingleObject with
INFINITE, so a hung script pinned the command worker permanently.
The agent already reported agentversion but the interactive tray app, which ships and updates independently, had no reported version. The agent now reads the tray exe file version and reports it as trayversion; EndpointState caches it (migration 0011) and the Fleet card surfaces it beside Agent. The tray csproj gains an explicit Version so the file version is stamped. Groundwork for per-PC version surfacing and push-update targeting.
Ben Baker's installed-software collection, software delivery (run_script and
update_app), agentless adoption tool and command-queue hardening, with his
authorship preserved.
Reviewed before promotion. Fixed: the endpoints plugin tests were never collected
by pytest, the SBOM snapshot table's key would exhaust on MySQL, ingest did not
truncate two columns strict MySQL would reject, and update_app's version branch
pinned a digest for a file that route never serves. Agent packaging moved off
require_role('admin') onto a new endpoints.agentpackage permission.
Contract 0.27.0. Full suite 2231, frontend 314.
Applied on top of af13337 so nothing already on wilm is rewritten.
AUTHORIZATION. The six new policy GETs were @jwt_required(optional=True) with
no permission at all, so an anonymous caller could list configuration and
business policies - gated only by a feature flag that 404s while off. They now
require computers.view. The six policy write routes stacked @jwt_required()
above @require_permission, the same pattern removed from nine endpoints routes
in this very push: the outer decorator answers an anonymous request 401 before
require_permission can consult the guest role, so a grant to guests silently
does nothing. Removed, and the now-unused import with them.
tests/test_core/test_public_read_surface.py failed at af13337 with seven ungated
GETs. It passes now. The seventh, endpoints.claim_next_command, is genuinely
agent-facing - authenticated in the handler by the endpoints collector key,
because a bay has no user session - so it joins PUBLIC_ENDPOINTS with its reason
written down, exactly as report_command_result already does in test_authz.py.
Note which half is public: claiming a queued command is the agent's business,
while ENQUEUEING one is the real authority and stays gated on endpoints.command.
INSTALLER. -BaseUrl and -CollectorKey are now mandatory with no defaults, and
https is enforced unless -AllowInsecureHttp is passed for a local test box. The
defaults were a real hostname (ADR-015) and a live collector key: a key in
source is a key in git history, and that key is also the command channel's kill
switch. Removing it here does not undo the disclosure - it must be rotated on
the server, and purged from af13337 - but it stops the next clone carrying it.
IMPORT. A committed import wrote thousands of rows and left no audit trail; it
now writes one AuditLog row inside the same transaction, so it cannot survive a
rollback or go missing from a commit. The computer-type dropdown asked for
per_page: 200, which the API ignores in favour of perpage and clamps to 100
anyway - it uses a new computersApi.types.listAll(), the pattern already applied
to the other selectors. And the comment claiming the template example rows are
"emitted as commented rows the engine skips" was the opposite of the truth: they
import as live data, the emitted instructions say so, and a test depends on it.
Not fixed here, deliberately: rotating the leaked key, purging it from history,
and the endpoint agent's own hardening (per-host credentials, a protected
process denylist so stop cannot kill security tooling, signing before the
PowerShell runner is enabled, registry ACLs, and the 25-second long poll that
will pin a worker thread per bay).
A command reaching 'succeeded' only proves the agent ran the action, not that it posted a new telemetry snapshot. Reloading panels on command success alone re-rendered stale service/process rows, so a stopped process could still show running right after Refresh Status.
Add a lightweight telemetrymeta probe exposing lastreportedat and gate both the header Refresh Status action and inline row Start/Stop on that stamp advancing past a pre-action baseline. Polling is bounded so an offline or idle agent never hangs the button, and messaging distinguishes a confirmed refresh from a queued one.
Bring the .NET endpoint agent source (service, tray, installer) into the repo so Cameron can merge and test, with an agent/.gitignore keeping bin/obj/publish out. Includes the DncMain launch-path fix, tray single-file host env scrub, and best-effort file logger.
