ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add the enrollment page, move scripts to per-PC credentials, and document the rollout

Browse Source 
- Settings > Endpoints > Enrollment. Migration tab: readiness counts and
  progress, each PC's state with the reasons it is blocked, "Upgrade
  selected" and "Upgrade next 25", and live wave progress. Credentials tab:
  enrolled PCs with revoke, pending requests with approve and reject, and the
  shared key and auto-enroll switches. Turning the shared key off asks for
  confirmation and shows how many PCs are not enrolled.
- The server certificate page shows the SPKI pin to put in the GE-Enforce
  pin entry.
- The agent upgrade script and the asset and printer reporters send the PC's
  credential when it has one (over https, only to the issuing server), fall
  back to the shared key, check server pins, and log refusals clearly.
- New Set-ShopDBServerPins.ps1 (GE-Enforce entry) and Get-ShopDBServerPin.ps1.
- AGENT-CONTRACT.md documents per-PC credentials; the rollout runbook covers
  phases, gates, rollback, pin and wave entries, and every place the shared
  key lives and how to clear it at cutover.
- Removed the 09-14 certificate design notes and dated session reports, and
  two VM notes that carried a live collector key and advised plain http.
- Corrected the GE-Enforce collector note, which claimed a plain registry
  token is protected by ACL.
This commit is contained in:
cproudlock
2026-09-15 12:04:12 -04:00
parent 35c17d2ea2
commit 1164cfb287
26 changed files with 2769 additions and 4058 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.github/GE-ENFORCE-COLLECTOR-FIX.html vendored
View File

@@ -374,7 +374,10 @@ try {
Set-ItemProperty -Path $RegPath -Name 'BaseUrl' -Value $ShopdbBaseUrl -Type String
Write-Output "Set BaseUrl: $ShopdbBaseUrl"
# Set ApiToken (REG_SZ - plain text, registry ACL protects it)
# Set ApiToken (REG_SZ - plain text). HKLM\SOFTWARE\GE\ShopDB inherits
# read for BUILTIN\Users, so any logged-on user can read this value unless
# an ACL is applied to the key. Per-PC credentials remove the need for a
# fleet secret here: see plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md
Set-ItemProperty -Path $RegPath -Name 'ApiToken' -Value $ShopdbApiToken -Type String
# Verify token was written

BIN
AGENT-FINGERPRINT-IMPLEMENTATION-2026-09-14.md
View File

Binary file not shown.

695
CERT-AUTH-IMPLEMENTATION.md
View File

@@ -1,695 +0,0 @@
# Per-Endpoint Certificate Authentication - Implementation Guide
## Overview
Implement client certificate authentication for ShopDB endpoint agents that works across multiple deployment types:
- **WJ Production:** IIS 2019 on Windows Server
- **Other Sites:** Docker Compose with Nginx reverse proxy
## Architecture
```
Agent (has cert) → Server (IIS or Nginx) → Flask
↓
Extracts cert
Passes to Flask via header/environ
↓
Flask validates fingerprint
against database
```
## Phase 1: Database Schema
Add client certificate columns to `assets` table.
**Migration:** `plugins/endpoints/migrations/versions/0021_client_certificates.py`
```python
"""Add client certificate authentication columns
Revision ID: endpoints0021clientcerts
Revises: endpoints0020appfingerprints
Create Date: 2026-09-14
"""
from alembic import op
import sqlalchemy as sa
revision = 'endpoints0021clientcerts'
down_revision = 'endpoints0020appfingerprints'
branch_labels = None
depends_on = None
plugin_name = 'endpoints'
def upgrade():
# Add client cert columns to assets table
op.add_column('assets',
sa.Column('client_cert_fingerprint', sa.String(64), nullable=True))
op.add_column('assets',
sa.Column('cert_issued_at', sa.DateTime, nullable=True))
op.add_column('assets',
sa.Column('cert_expires_at', sa.DateTime, nullable=True))
# Add index for fast fingerprint lookup during authentication
op.create_index(
'idx_assets_cert_fingerprint',
'assets',
['client_cert_fingerprint'],
unique=False
)
def downgrade():
op.drop_index('idx_assets_cert_fingerprint', 'assets')
op.drop_column('assets', 'cert_expires_at')
op.drop_column('assets', 'cert_issued_at')
op.drop_column('assets', 'client_cert_fingerprint')
```
**Run migration:**
```bash
docker compose exec api flask db upgrade
```
---
## Phase 2: Flask Code (Universal)
Add client certificate authentication support that works with both IIS and Nginx.
**File:** `shopdb/core/auth/client_cert.py` (new file)
```python
"""Client certificate authentication helpers.
Extracts client certificate fingerprints from various deployment types:
- IIS 2019: SSL_CLIENT_CERT environ (PEM format)
- IIS with ARR: X-ARR-ClientCert header (base64 DER)
- Nginx: X-Client-Cert-Fingerprint header (pre-computed)
"""
import hashlib
import base64
from flask import request, current_app
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
def get_client_cert_fingerprint():
"""Extract client certificate SHA256 fingerprint from request.
Returns:
str: 64-character lowercase hex fingerprint, or None if no cert
Supports multiple deployment types:
- Nginx: reads X-Client-Cert-Fingerprint header (SHA256, colon-separated)
- IIS direct: reads SSL_CLIENT_CERT environ (PEM format)
- IIS with ARR: reads X-ARR-ClientCert header (base64 DER)
"""
# Option 1: Nginx passes pre-computed fingerprint
# Format: "AB:CD:EF:..." (colon-separated uppercase hex)
nginx_fp = request.headers.get('X-Client-Cert-Fingerprint')
if nginx_fp:
# Normalize: remove colons, lowercase
return nginx_fp.replace(':', '').lower()
# Option 2: IIS passes full cert in PEM format via WSGI environ
# This is the standard IIS behavior when client certs are enabled
iis_cert_pem = request.environ.get('SSL_CLIENT_CERT')
if iis_cert_pem:
try:
# Parse PEM certificate
if isinstance(iis_cert_pem, str):
iis_cert_pem = iis_cert_pem.encode('utf-8')
cert = x509.load_pem_x509_certificate(iis_cert_pem, default_backend())
# Convert to DER and hash
cert_der = cert.public_bytes(serialization.Encoding.DER)
fingerprint = hashlib.sha256(cert_der).hexdigest()
current_app.logger.debug(f'[client_cert] Extracted from IIS SSL_CLIENT_CERT: {fingerprint[:16]}...')
return fingerprint
except Exception as e:
current_app.logger.warning(f'[client_cert] Failed to parse IIS cert: {e}')
return None
# Option 3: IIS with ARR (Application Request Routing) passes base64-encoded DER
arr_cert_b64 = request.headers.get('X-ARR-ClientCert')
if arr_cert_b64:
try:
# Decode base64 to get DER bytes
cert_der = base64.b64decode(arr_cert_b64)
fingerprint = hashlib.sha256(cert_der).hexdigest()
current_app.logger.debug(f'[client_cert] Extracted from ARR header: {fingerprint[:16]}...')
return fingerprint
except Exception as e:
current_app.logger.warning(f'[client_cert] Failed to parse ARR cert: {e}')
return None
# No client certificate present
return None
def authenticate_by_client_cert():
"""Authenticate request by client certificate.
Returns:
Asset: The authenticated asset, or None if cert auth fails
Checks:
1. Client cert present in request?
2. Fingerprint matches an asset in database?
3. Certificate not expired?
Does NOT raise exceptions - returns None on any failure.
Caller decides whether to fall back to other auth methods.
"""
from shopdb.api import Asset
from datetime import datetime, timezone
fingerprint = get_client_cert_fingerprint()
if not fingerprint:
return None
# Look up asset by cert fingerprint
asset = Asset.query.filter_by(client_cert_fingerprint=fingerprint).first()
if not asset:
current_app.logger.warning(
f'[client_cert] Unknown fingerprint: {fingerprint[:16]}... '
f'(not associated with any asset)'
)
return None
# Check certificate expiration
if asset.cert_expires_at:
now = datetime.now(timezone.utc).replace(tzinfo=None)
if asset.cert_expires_at < now:
current_app.logger.warning(
f'[client_cert] Expired certificate for asset {asset.assetid} '
f'(expired {asset.cert_expires_at})'
)
return None
current_app.logger.info(
f'[client_cert] Authenticated asset {asset.assetid} '
f'via certificate {fingerprint[:16]}...'
)
return asset
```
**File:** `plugins/endpoints/plugin.py` (modify `apply_collector_payload`)
```python
# Add import at top
from shopdb.core.auth.client_cert import authenticate_by_client_cert
# In apply_collector_payload method, BEFORE existing auth check:
def apply_collector_payload(self, payload: Dict) -> Dict:
"""Process endpoint collector payload (ADR-006 contract)."""
# CERT AUTH FIRST: Check for client certificate
cert_asset = authenticate_by_client_cert()
if cert_asset:
# Client cert authenticated - use this asset
assetid = cert_asset.assetid
hostname = payload.get('hostname', 'UNKNOWN')
current_app.logger.info(
f'[collector] asset_id={assetid} authenticated via client certificate'
)
# Update cert metadata if agent provides new expiration
if 'cert_expires_at' in payload:
cert_asset.cert_expires_at = payload['cert_expires_at']
db.session.flush()
else:
# FALLBACK: Existing CollectorKey authentication
# (your existing _collector_auth_or_error logic here)
pass
# Rest of method unchanged...
```
---
## Phase 3: IIS 2019 Configuration (WJ Production)
### Step 1: Enable Client Certificates in IIS
1. Open **IIS Manager**
2. Select your ShopDB site (not server level)
3. Open **SSL Settings**
4. Check **"Accept"** (not "Require" - we want dual-auth during rollout)
5. Click **Apply**
### Step 2: Verify Certificate Mapping (Optional)
IIS will automatically pass client certs to WSGI via `SSL_CLIENT_CERT` environ variable - no additional config needed.
If using ARR (Application Request Routing):
1. Open **URL Rewrite** for your site
2. Add **Server Variable**: `HTTP_X_ARR_ClientCert`
3. Set value to: `{HTTP_SSL_CLIENT_CERT}`
### Step 3: Test with curl
```powershell
# Generate test client cert (PowerShell)
$cert = New-SelfSignedCertificate `
-Subject "CN=TEST-ENDPOINT" `
-CertStoreLocation "Cert:\CurrentUser\My" `
-KeyExportPolicy Exportable `
-NotAfter (Get-Date).AddYears(2)
# Export to PFX
$pwd = ConvertTo-SecureString -String "test123" -Force -AsPlainText
Export-PfxCertificate `
-Cert $cert `
-FilePath "C:\temp\test-client.pfx" `
-Password $pwd
# Test with curl (Git Bash on Windows)
curl -k --cert-type P12 --cert C:/temp/test-client.pfx:test123 \
https://localhost/api/endpoints/test/client-cert
```
**Expected response:**
```json
{
"data": {
"message": "Client cert test endpoint",
"ssl_client_cert_present": true,
"fingerprint_sha256": "325e42ef..."
}
}
```
### Step 4: Configure Agent to Use Certificate
**Agent change (C# .NET):**
```csharp
// ShopDbEndpointAgent.Core/EndpointCollector.cs
private static HttpClient CreateHttpClient(string certThumbprint = null)
{
var handler = new HttpClientHandler
{
ServerCertificateCustomValidationCallback = (msg, cert, chain, errors) =>
{
// Accept self-signed server cert (existing behavior)
return true;
}
};
// If client cert configured, attach it
if (!string.IsNullOrEmpty(certThumbprint))
{
var cert = GetClientCertFromStore(certThumbprint);
if (cert != null)
{
handler.ClientCertificates.Add(cert);
Log.Information($"Using client certificate: {cert.Subject}");
}
}
return new HttpClient(handler)
{
Timeout = TimeSpan.FromSeconds(90)
};
}
private static X509Certificate2 GetClientCertFromStore(string thumbprint)
{
using (var store = new X509Store(StoreName.My, StoreLocation.LocalMachine))
{
store.Open(OpenFlags.ReadOnly);
var certs = store.Certificates.Find(
X509FindType.FindByThumbprint,
thumbprint,
validOnly: false
);
return certs.Count > 0 ? certs[0] : null;
}
}
```
**Config file:** `C:\ProgramData\ShopDB\config.json`
```json
{
"apiUrl": "https://shopdb.wj.local/api",
"clientCertThumbprint": "325E42EFA401CDA4B75BB7B478CFE01FBD5EE956C629ADD3A075A74847D71F77"
}
```
---
## Phase 4: Docker + Nginx Configuration (Other Sites)
### Step 1: Add nginx.conf
**File:** `nginx.conf` (create in repo root)
```nginx
events {
worker_connections 2048;
}
http {
upstream flask_backend {
server api:5001;
}
server {
listen 443 ssl;
server_name _;
# Server certificate (HTTPS)
ssl_certificate /etc/nginx/certs/server.crt;
ssl_certificate_key /etc/nginx/certs/server.key;
# Client certificate validation (optional during rollout)
# optional_no_ca: accept cert but don't validate against CA
# Flask validates fingerprint against database
ssl_verify_client optional_no_ca;
# Increase timeouts for long-polling agents
proxy_connect_timeout 90s;
proxy_send_timeout 90s;
proxy_read_timeout 90s;
# Health check endpoint
location /api/dashboard/health {
proxy_pass http://flask_backend;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
# All other routes
location / {
proxy_pass http://flask_backend;
proxy_http_version 1.1;
proxy_set_header Connection "";
# Pass real client IP (not nginx IP)
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
# Pass client certificate fingerprint to Flask
# $ssl_client_fingerprint format: "AB:CD:EF:..." (colon-separated hex)
proxy_set_header X-Client-Cert-Fingerprint $ssl_client_fingerprint;
proxy_set_header X-Client-Cert-S-DN $ssl_client_s_dn;
}
# Access log includes client cert status
access_log /dev/stdout combined;
error_log /dev/stderr warn;
}
}
```
### Step 2: Update docker-compose.yml
```yaml
services:
nginx:
image: nginx:1.27-alpine
restart: unless-stopped
depends_on:
- api
ports:
- "5001:443"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./certs:/etc/nginx/certs:ro
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "--no-check-certificate", "https://localhost/api/dashboard/health"]
interval: 30s
timeout: 5s
retries: 3
api:
# ... existing config ...
ports:
# Expose on localhost only (for debugging)
- "127.0.0.1:5002:5001"
command:
- gunicorn
- --bind
- 0.0.0.0:5001
# REMOVE: --certfile (nginx does TLS now)
# REMOVE: --keyfile
# REMOVE: --cert-reqs
- --worker-class
- gevent
- --workers
- "4"
- --worker-connections
- "2000"
- --timeout
- "60"
- --access-logfile
- "-"
- --access-logformat
- '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s %(D)sus "%(f)s" "%(a)s"'
- --error-logfile
- "-"
- wsgi:app
# Remove healthcheck (nginx handles it now)
```
### Step 3: Rebuild and Deploy
```bash
# Stop current stack
docker compose down
# Start with nginx
docker compose up -d
# Verify nginx is proxying
curl -k https://localhost:5001/api/dashboard/health
# Test with client cert (same cert from earlier test)
openssl s_client -connect localhost:5001 \
-cert certs/test-client.crt \
-key certs/test-client.key \
-quiet <<< "GET /api/endpoints/test/client-cert HTTP/1.1
Host: localhost:5001
Connection: close
"
```
---
## Phase 5: Agent Certificate Generation
### PowerShell Script (Windows agents)
**File:** `scripts/generate-agent-cert.ps1`
```powershell
param(
[Parameter(Mandatory=$true)]
[string]$Hostname,
[Parameter(Mandatory=$false)]
[int]$ValidityYears = 2
)
# Generate self-signed cert for this endpoint
$cert = New-SelfSignedCertificate `
-Subject "CN=$Hostname" `
-DnsName $Hostname `
-CertStoreLocation "Cert:\LocalMachine\My" `
-KeyExportPolicy NonExportable `
-KeyUsage DigitalSignature, KeyEncipherment `
-NotAfter (Get-Date).AddYears($ValidityYears) `
-HashAlgorithm SHA256
Write-Host "Generated certificate:"
Write-Host " Thumbprint: $($cert.Thumbprint)"
Write-Host " Subject: $($cert.Subject)"
Write-Host " Expires: $($cert.NotAfter)"
# Compute SHA256 fingerprint (for database registration)
$certBytes = $cert.Export([System.Security.Cryptography.X509Certificates.X509ContentType]::Cert)
$sha256 = [System.Security.Cryptography.SHA256]::Create()
$fingerprintBytes = $sha256.ComputeHash($certBytes)
$fingerprint = ($fingerprintBytes | ForEach-Object { $_.ToString("x2") }) -join ""
Write-Host ""
Write-Host "Register in ShopDB with:"
Write-Host " UPDATE assets"
Write-Host " SET client_cert_fingerprint = '$fingerprint',"
Write-Host " cert_issued_at = CURRENT_TIMESTAMP,"
Write-Host " cert_expires_at = '$($cert.NotAfter.ToString("yyyy-MM-dd HH:mm:ss"))'"
Write-Host " WHERE hostname = '$Hostname';"
# Update agent config
$configPath = "C:\ProgramData\ShopDB\config.json"
if (Test-Path $configPath) {
$config = Get-Content $configPath | ConvertFrom-Json
$config.clientCertThumbprint = $cert.Thumbprint
$config | ConvertTo-Json | Set-Content $configPath
Write-Host ""
Write-Host "Updated $configPath with thumbprint"
}
return $cert.Thumbprint
```
**Usage:**
```powershell
# On endpoint VM
.\generate-agent-cert.ps1 -Hostname "DESKTOP-HF1K9M4"
# Output:
# Generated certificate:
# Thumbprint: 325E42EFA401CDA4B75BB7B478CFE01FBD5EE956C629ADD3A075A74847D71F77
# Subject: CN=DESKTOP-HF1K9M4
# Expires: 9/14/2028 12:00:00 PM
#
# Register in ShopDB with:
# UPDATE assets
# SET client_cert_fingerprint = '325e42efa401cda4b75bb7b478cfe01fbd5ee956c629add3a075a74847d71f77',
# cert_issued_at = CURRENT_TIMESTAMP,
# cert_expires_at = '2028-09-14 12:00:00'
# WHERE hostname = 'DESKTOP-HF1K9M4';
```
---
## Rollout Sequence
### Week 1: WJ Production (IIS)
1. Deploy Flask code (cert auth + CollectorKey fallback)
2. Configure IIS SSL Settings → "Accept" client certs
3. Generate cert for 1 test endpoint
4. Register cert fingerprint in database
5. Update agent config with thumbprint
6. Verify dual-auth works (cert + fallback)
### Week 2: Remaining WJ Fleet
1. Generate certs for remaining endpoints (PowerShell script)
2. Register fingerprints in database
3. Deploy updated agent configs
### Week 3: Docker Sites
1. Add nginx container to docker-compose.yml
2. Deploy updated bundle to 1 pilot site
3. Generate certs for that site's endpoints
4. Register fingerprints
5. Verify nginx proxy works
### Month 2: Remaining Sites
1. Roll out nginx updates to remaining Docker sites
2. Generate certs per-site
3. After 100% cert adoption: change `ssl_verify_client optional` to `require`
---
## Testing Checklist
### Flask Tests
- [ ] `get_client_cert_fingerprint()` returns None when no cert
- [ ] `get_client_cert_fingerprint()` parses nginx header (X-Client-Cert-Fingerprint)
- [ ] `get_client_cert_fingerprint()` parses IIS environ (SSL_CLIENT_CERT)
- [ ] `authenticate_by_client_cert()` returns asset when fingerprint matches
- [ ] `authenticate_by_client_cert()` returns None when cert expired
- [ ] Collector falls back to CollectorKey when cert auth fails
### IIS Tests
- [ ] IIS passes `SSL_CLIENT_CERT` to Flask
- [ ] Agent with cert authenticates successfully
- [ ] Agent without cert falls back to CollectorKey
- [ ] Expired cert rejected
### Docker/Nginx Tests
- [ ] Nginx forwards requests to Gunicorn
- [ ] Nginx adds `X-Client-Cert-Fingerprint` header
- [ ] Agent with cert authenticates via nginx
- [ ] Agent without cert uses CollectorKey
- [ ] Long-polling works (25+ second holds)
---
## Security Considerations
1. **Cert Storage:** LocalMachine\My store (admin-only access)
2. **Private Key:** Non-exportable (can't be stolen)
3. **Expiration:** 2 years (balance security vs. rotation overhead)
4. **Revocation:** Remove fingerprint from database (immediate)
5. **Self-Signed:** No CA needed (air-gapped sites)
---
## Rollback Plan
### If Flask code breaks:
```bash
git revert <commit>
docker compose restart api # or iisreset
```
### If IIS config breaks:
1. IIS Manager → SSL Settings → Uncheck "Accept"
2. Click Apply
3. Agents fall back to CollectorKey immediately
### If nginx breaks:
```bash
# Restore direct Gunicorn access
docker compose down nginx
# Re-add --certfile/--keyfile to api service
docker compose up -d api
```
---
## Monitoring
### Metrics to Track
- % of agents using cert auth vs. CollectorKey
- Cert expiration warnings (30 days before)
- Failed cert authentications (unknown fingerprint)
- Expired cert attempts
### Logs to Watch
```
[client_cert] Authenticated asset 2528 via certificate 325e42ef...
[client_cert] Unknown fingerprint: 325e42ef... (not associated with any asset)
[client_cert] Expired certificate for asset 2528 (expired 2026-09-14 12:00:00)
```
---
## Next Steps
1. **Create migration** (0021_client_certificates.py)
2. **Add Flask auth code** (client_cert.py)
3. **Test on IIS** (your WJ production)
4. **Add nginx config** (for Docker sites)
5. **Generate first cert** (test endpoint)
6. **Document in wiki** (deployment runbook)
**Estimated effort:**
- Flask code: 2 hours
- IIS config + test: 1 hour
- Docker/nginx: 1 hour
- Agent changes: 2 hours
- **Total: 6 hours**
**Ready to start with Flask code?**

450
CERT-AUTH-SECURITY.md
View File

@@ -1,450 +0,0 @@
# Client Certificate Authentication Security Documentation
## Overview
ShopDB endpoint agent uses mTLS (mutual TLS) for per-endpoint authentication. This document covers security considerations, deployment requirements, and operational procedures.
**Last Updated:** 2026-09-14
**Status:** Production-ready with documented limitations
---
## Security Model
### Authentication Flow
1. **Agent Startup**: Loads client certificate from LocalMachine\My by thumbprint
2. **HTTPS Connection**: Presents certificate to nginx during TLS handshake
3. **Nginx Validation**: Accepts certificate (no CA validation - self-signed OK)
4. **Fingerprint Extraction**: Nginx computes SHA1, passes to Flask via header
5. **Flask Authentication**: Looks up asset by fingerprint, checks revocation/expiration
6. **Authorization**: Verifies cert's asset matches payload's hostname asset
7. **Fallback**: If cert auth fails, falls back to collector API key
### Dual-Auth During Rollout
- **Phase 1-3**: Both cert AND API key accepted (backward compatible)
- **Phase 4**: API key disabled via `plugin.endpoints.envkeyfallback=false`
- **Rollback**: Re-enable API key fallback instantly if cert auth breaks
---
## Known Limitations
### 1. SHA1 Fingerprints (nginx limitation)
**Issue:** Nginx `$ssl_client_fingerprint` outputs SHA1 (40 hex chars), not SHA256.
**Why SHA1?**
- Nginx does not expose SHA256 fingerprint via built-in variables
- Would require custom nginx module or full certificate parsing in Flask
**Is SHA1 Acceptable?**
- ✅ YES for certificate fingerprinting (not signing)
- ✅ Acceptable in air-gapped environments with:
- Short-lived certificates (2 years max)
- Certificate revocation support (cert_revoked_at column)
- Self-signed certificates only (no external CA trust)
**SHA1 Collision Risk:**
- Computationally expensive to generate collision (estimated $100K+ in 2026)
- Would require attacker to generate cert with same SHA1 as legitimate endpoint
- Air-gapped environment prevents external certificate injection
- Revocation support allows instant kill switch if compromised
**Migration to SHA256:**
```
Option 1: Custom nginx module to expose SHA256
Option 2: Migrate to IIS (exposes full PEM cert for server-side hashing)
Option 3: Pass full certificate to Flask for SHA256 computation (bandwidth cost)
```
### 2. Private Key Permissions (tray access)
**Issue:** PowerShell script grants `BUILTIN\Users` read access to private key.
**Why?** Tray app (running as interactive user) needs cert access for manual agent runs.
**Risk:** ANY interactive user on the machine can extract the private key.
**Mitigations:**
1. **Best:** Remove tray dependency (tray calls service API, service uses cert)
2. **Acceptable:** Grant only to specific user account running tray
3. **Current:** Document that tray deployment requires trusted console users only
**When Acceptable:**
- Shop floor PCs with single trusted operator per machine
- Kiosks where all users are authorized operators
- Environments where console access = trusted
**NOT Acceptable:**
- Multi-user workstations with untrusted users
- Shared administrator workstations
- Development machines
### 3. No Certificate Revocation List (CRL)
**Issue:** No automated CRL distribution. Revocation is manual SQL UPDATE.
**Workaround:**
```sql
-- Revoke certificate (takes effect immediately)
UPDATE assets
SET cert_revoked_at = CURRENT_TIMESTAMP,
cert_revocation_reason = 'Compromised - laptop stolen - TICKET-456'
WHERE client_cert_fingerprint = 'abc123...';
```
**Future:** Web UI for certificate management (provision, revoke, monitor expiration).
---
## Security Features Implemented
### ✅ Constant-Time Fingerprint Comparison
Prevents timing attacks on fingerprint validation:
```python
if not hmac.compare_digest(asset.client_cert_fingerprint, fingerprint):
return None # Constant time, no early exit
```
### ✅ Format Validation
Rejects malformed fingerprints before database lookup:
```python
# Must be 40 hex chars (SHA1) or 64 hex chars (SHA256)
if len(normalized) not in (40, 64):
return None
if not re.match(r'^[0-9a-f]+$', normalized):
return None
```
### ✅ Certificate Revocation Support
Admin can revoke compromised certificates instantly:
```sql
-- Revoke
UPDATE assets SET cert_revoked_at = NOW(), cert_revocation_reason = '...' WHERE ...;
-- Un-revoke (if cert was revoked by mistake)
UPDATE assets SET cert_revoked_at = NULL, cert_revocation_reason = NULL WHERE ...;
```
### ✅ Expiration Enforcement
Expired certificates rejected automatically:
```python
if asset.cert_expires_at < now:
return None # Must regenerate and re-register
```
### ✅ Authorization Check
Certificate authenticates asset X, but payload reports for asset Y → DENIED:
```python
if g.cert_authenticated_asset_id != assetid:
raise PermissionError(...) # Prevents cross-asset impersonation
```
### ✅ Audit Logging
All authentication attempts logged to AuditLog:
- `auth.cert.success` - Successful authentication
- `auth.cert.unknown` - Unknown fingerprint
- `auth.cert.revoked` - Attempt to use revoked cert
- `auth.cert.expired` - Attempt to use expired cert
---
## Deployment Requirements
### Phase 1: Server Deployment
1. **Deploy nginx with mTLS support:**
```yaml
# docker-compose.yml
services:
nginx:
image: nginx:alpine
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ./certs:/etc/nginx/certs
```
2. **Run database migrations:**
```bash
# Adds client_cert_fingerprint, cert_issued_at, cert_expires_at,
# cert_revoked_at, cert_revocation_reason columns
flask db upgrade
```
3. **Verify fallback still works:**
```bash
# Existing agents continue using API key
curl -H "X-API-Key: $COLLECTOR_API_KEY" https://shopdb/api/collector/endpoints
```
### Phase 2: Certificate Generation (per endpoint)
1. **Run PowerShell script on endpoint:**
```powershell
.\generate-agent-cert.ps1 -ValidityYears 2
```
2. **Script outputs SQL to register certificate:**
```sql
UPDATE assets
SET client_cert_fingerprint = 'abc123...',
cert_issued_at = CURRENT_TIMESTAMP,
cert_expires_at = '2028-09-14 12:00:00',
cert_revoked_at = NULL,
cert_revocation_reason = NULL
WHERE assetnumber = 'G5QX1GT3ESF';
```
3. **Verify agent config updated:**
```json
{
"baseurl": "https://shopdb.company.com",
"apikey": "collector_key_here",
"clientCertThumbprint": "ABC123...",
"cadenceseconds": 900
}
```
4. **Restart agent service:**
```powershell
Restart-Service ShopDbEndpointAgent
```
5. **Verify cert auth in logs:**
```
[client_cert] Authenticated asset 1234 (number=G5QX1GT3ESF) via certificate abc123...
```
### Phase 3: Fleet Rollout
**Automation Required:**
- PowerShell script to generate certs across fleet
- CSV export of asset_number,fingerprint,expires_at
- SQL bulk INSERT script
- Group Policy or deployment tool to push agent config updates
**Validation:**
- Monitor `auth.cert.success` audit log entries
- Dashboard showing cert expiration dates
- Alert on endpoints still using API key after N days
### Phase 4: Retire API Key Fallback
**Only after 100% cert coverage verified:**
```sql
UPDATE settings
SET value = 'false'
WHERE key = 'plugin.endpoints.envkeyfallback';
```
**Revoke old collector API key tokens:**
```python
# In Flask admin UI or API
revoke_service_tokens('collector.ingest.endpoints')
```
**CRITICAL:** Test rollback plan before this step!
---
## Operational Procedures
### Certificate Rotation
**When to rotate:**
- 30 days before expiration (automated alert recommended)
- After suspected compromise
- After employee termination (if they had console access)
**How to rotate:**
1. Generate new certificate (keep old one installed temporarily)
2. Update database with new fingerprint
3. Update agent config with new thumbprint
4. Restart agent
5. Verify new cert working
6. Remove old cert from certificate store
### Certificate Revocation
**Immediate revocation (compromised/lost endpoint):**
```sql
UPDATE assets
SET cert_revoked_at = CURRENT_TIMESTAMP,
cert_revocation_reason = 'Laptop stolen - INCIDENT-789'
WHERE assetnumber = 'G5QX1GT3ESF';
```
**Effect:** Agent's next request denied, falls back to API key if enabled.
### Monitoring
**Key metrics to monitor:**
1. **Cert Expiration:**
```sql
SELECT assetnumber, cert_expires_at,
DATEDIFF(cert_expires_at, NOW()) AS days_remaining
FROM assets
WHERE cert_expires_at IS NOT NULL
AND cert_expires_at < DATE_ADD(NOW(), INTERVAL 30 DAY)
ORDER BY cert_expires_at;
```
2. **Auth Failure Rate:**
```sql
SELECT action, COUNT(*) AS count
FROM auditlog
WHERE action LIKE 'auth.cert.%'
AND createdat > NOW() - INTERVAL 24 HOUR
GROUP BY action;
```
3. **Endpoints Still Using API Key:**
```sql
SELECT e.hostname, e.lastreportedat
FROM endpointstates e
JOIN assets a ON a.assetid = e.assetid
WHERE a.client_cert_fingerprint IS NULL
AND e.lastreportedat > NOW() - INTERVAL 1 HOUR;
```
---
## Troubleshooting
### Agent reports but logs show API key auth (not cert)
**Cause:** Client certificate not configured or not found.
**Fix:**
```powershell
# Verify cert exists
Get-ChildItem Cert:\LocalMachine\My | Where-Object { $_.Subject -like "*$env:COMPUTERNAME*" }
# Verify config
Get-Content C:\ProgramData\ShopDB\config.json | ConvertFrom-Json
# Verify cert has private key
$cert = Get-Item "Cert:\LocalMachine\My\$thumbprint"
$cert.HasPrivateKey # Should be True
```
### "Revoked certificate" in logs but cert not revoked
**Cause:** Database cert_revoked_at column set incorrectly.
**Fix:**
```sql
UPDATE assets
SET cert_revoked_at = NULL,
cert_revocation_reason = NULL
WHERE assetnumber = 'G5QX1GT3ESF';
```
### "Unknown fingerprint" in logs
**Cause:** Certificate fingerprint not registered in database.
**Fix:**
```powershell
# Get fingerprint from cert
$cert = Get-Item "Cert:\LocalMachine\My\$thumbprint"
$certBytes = $cert.Export([System.Security.Cryptography.X509Certificates.X509ContentType]::Cert)
$sha1 = [System.Security.Cryptography.SHA1]::Create()
$fp = ($sha1.ComputeHash($certBytes) | ForEach-Object { $_.ToString("x2") }) -join ""
# Register in database
UPDATE assets SET client_cert_fingerprint = '$fp' WHERE assetnumber = 'G5QX1GT3ESF';
```
### Agent fails after cert rotation
**Cause:** Old cert removed before new cert registered in database.
**Fix:**
1. Reinstall old cert temporarily
2. Register new cert fingerprint in database
3. Update agent config to new thumbprint
4. Restart agent
5. Remove old cert
---
## Security Best Practices
### ✅ DO
- Generate certificates with 2-year maximum validity
- Rotate certificates 30 days before expiration
- Revoke certificates immediately when endpoint compromised
- Monitor audit logs for revoked/expired cert attempts
- Document certificate issuance and revocation in tickets
- Test rollback to API key before disabling fallback
- Use short-lived certificates in untrusted environments
### ❌ DON'T
- Grant `BUILTIN\Users` access to private keys on multi-user machines
- Skip hostname validation in cert generation script
- Deploy to production without testing cert revocation
- Disable API key fallback without verified 100% cert coverage
- Use expired certificates (regenerate immediately)
- Share certificates between endpoints
- Export private keys to network shares
---
## Future Enhancements
1. **Web UI for Certificate Management**
- Upload CSRs or paste fingerprints
- View expiration dashboard
- One-click revocation
- Automated renewal reminders
2. **SHA256 Migration**
- Custom nginx module for SHA256
- Or migrate to IIS deployment
- Dual-column support (SHA1 + SHA256) during migration
3. **Automated Certificate Rotation**
- Agent auto-generates new cert 30 days before expiration
- Server auto-accepts new fingerprint after validating old cert
- Zero-touch renewal
4. **Per-User Certificate ACLs**
- Grant tray access to specific user account only
- Service creates runas token for tray HTTP requests
---
## References
- **ADR-006:** Collector Contract (dual-auth model)
- **ADR-016:** Credential Delivery (provisioning workflow)
- **AGENT-CONTRACT.md:** Wire protocol and authentication
- **nginx.conf:** mTLS configuration and SHA1 limitation
- **generate-agent-cert.ps1:** Certificate generation script
---
## Contact
For security questions or to report vulnerabilities:
- **Internal:** File ticket in JIRA under "Security" component
- **Emergency:** Page on-call security engineer via PagerDuty

159
CERT-AUTH-SERVER-COMPARISON.md
View File

@@ -1,159 +0,0 @@
# Client Certificate Authentication - WSGI Server Comparison
## Current Setup
- **Gunicorn 26.2.0** with gevent workers
- 4 workers x 2000 greenlets = 8000 concurrent connections
- Long-polling agents hold connections for 25+ seconds
- TLS termination in Gunicorn with --certfile/--keyfile
- **Problem**: Gunicorn doesn't expose client cert to Flask via WSGI environ
## Option 1: Hypercorn (ASGI)
**Pros:**
- Native client cert support via `ssl.SSLContext.wrap_socket()`
- Exposes client cert via ASGI scope['extensions']['tls']['client_cert_der']
- Modern async server (Trio/asyncio)
- Supports HTTP/2 and WebSockets
**Cons:**
- **ASGI, not WSGI** - Flask is WSGI, need asgiref adapter
- Different async model than gevent (asyncio instead of greenlets)
- Less mature for Flask production deployments
- Breaking change: entire stack needs rework
**Verdict:** ❌ Too invasive - requires migrating from WSGI to ASGI
---
## Option 2: Waitress (Pure Python WSGI)
**Pros:**
- ✅ Already in requirements.txt (line 698: waitress==3.0.2)
- ✅ Pure Python WSGI server (drop-in Gunicorn replacement)
- ✅ Production-ready (powers PyPI, Pyramid, etc.)
- ✅ Native Windows support
- Thread-based concurrency (not gevent, but configurable)
**Cons:**
- ❌ **No client certificate support** - waitress doesn't expose client certs to WSGI apps
- Uses threads instead of gevent greenlets (heavier for long-polling)
- Lower connection ceiling than gevent (4 threads vs 8000 greenlets)
**Verdict:** ❌ Doesn't solve the problem - no client cert support
---
## Option 3: Nginx Reverse Proxy (Current Recommendation)
**Pros:**
- ✅ Battle-tested client cert validation
- ✅ Keep Gunicorn + gevent (no code changes)
- ✅ Extracts cert fingerprint and passes via X-Client-Cert-Fingerprint header
- ✅ Optional validation with `ssl_verify_client optional_no_ca`
- ✅ Standard pattern for mTLS in Docker
**Cons:**
- Adds one more container (nginx:alpine ~30MB)
- Slightly more complex docker-compose.yml
**Verdict:** ✅ Best option - proven, minimal changes, keeps gevent
---
## Option 4: uWSGI
**Pros:**
- Full-featured WSGI server
- Supports client certs via `uwsgi_param SSL_CLIENT_CERT`
- Gevent plugin available
**Cons:**
- Complex configuration
- Deprecated (project in maintenance mode as of 2023)
- No active development
**Verdict:** ❌ Deprecated, avoid
---
## Option 5: Custom Gunicorn Worker (SSLGeventWorker)
**Approach**: Extend gevent worker to extract client cert from SSLSocket and inject into WSGI environ
**Pros:**
- Keep Gunicorn + gevent
- No reverse proxy needed
- ~50 lines of Python
**Cons:**
- Custom code to maintain
- Accesses Gunicorn internals (may break on upgrades)
- Less tested than Nginx approach
**Example:**
```python
# custom_worker.py
from gunicorn.workers.ggevent import GeventWorker
import ssl
class SSLGeventWorker(GeventWorker):
def handle_request(self, listener, req, client, addr):
# Extract client cert from SSLSocket
if isinstance(client, ssl.SSLSocket):
try:
cert_der = client.getpeercert(binary_form=True)
if cert_der:
req.environ['peercert'] = cert_der
except Exception:
pass
return super().handle_request(listener, req, client, addr)
```
```yaml
# docker-compose.yml
command:
- gunicorn
- --worker-class
- custom_worker.SSLGeventWorker
- --cert-reqs
- "1"
- wsgi:app
```
**Verdict:** ⚠️ Possible but fragile - use if Nginx is not acceptable
---
## Recommendation
**Go with Option 3: Nginx reverse proxy**
### Why:
1. **Proven pattern** - industry standard for mTLS in Docker
2. **Minimal risk** - no changes to Flask or Gunicorn config
3. **Keep gevent** - preserves your 8000 concurrent connection capacity
4. **Optional validation** - `ssl_verify_client optional_no_ca` supports dual-auth during rollout
5. **Easy rollback** - just remove nginx container
### Implementation:
- Add nginx:alpine container (30MB)
- 10-line nginx.conf
- No Python code changes (Flask reads `X-Client-Cert-Fingerprint` header)
- 5 minutes to implement
- Test immediately with same test cert
### Alternative (if Nginx rejected):
- Option 5: Custom Gunicorn worker (50 lines, fragile)
- Option 1: Migrate to Hypercorn + ASGI (days of work, high risk)
---
## Research Summary
**Tested:** Gunicorn with --cert-reqs 1 accepts client cert but doesn't expose to Flask
**Researched:** Waitress (no client cert), uWSGI (deprecated), Hypercorn (ASGI only)
**Validated:** Nginx reverse proxy is standard solution for mTLS with WSGI apps
**Decision Point:** Add nginx container or write custom Gunicorn worker?

127
CRITICAL-FIXES-2026-09-14.md
View File

@@ -1,127 +0,0 @@
# Critical Fixes Applied - 2026-09-14
**Status**: All 4 critical issues from code review FIXED
---
## Fix #1: Added UNIQUE Constraint to Fingerprint Index
**File**: `plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py`
**Line**: 62
**Problem**: Index allowed duplicate (assetid, app_fingerprint) pairs, breaking delta logic assumption that one fingerprint = one row per asset.
**Fix**: Added `unique=True` to index creation:
```python
op.create_index(
'idx_endpointapplications_fingerprint',
'endpointapplications',
['assetid', 'app_fingerprint'],
unique=True # ADDED
)
```
**Impact**: Prevents duplicate fingerprints, ensures delete count verification works correctly.
---
## Fix #2: Event Deduplication Now Uses Advisory Lock
**File**: `plugins/endpoints/plugin.py`
**Lines**: 1454-1476
**Problem**: Row-level locks (FOR UPDATE) only lock EXISTING rows. When both agents see zero matching events (first-time report), zero rows are locked, allowing both to INSERT duplicate events → IntegrityError → transaction rollback → data loss.
**Fix**: Added PostgreSQL advisory lock BEFORE dedup check:
```python
# Acquire asset-level lock before checking for duplicates
from sqlalchemy import text
db.session.execute(
text('SELECT pg_advisory_xact_lock(:lock_key)'),
{'lock_key': assetid}
)
# Now safe to check for duplicates (lock prevents concurrent INSERT)
rows = (db.session.query(...)
.filter(...)
.all()) # Removed with_for_update() - advisory lock is stronger
```
**Impact**: Eliminates TOCTOU race on first-time event reports. Lock held until transaction commit, ensuring only one agent can check-and-insert at a time.
---
## Fix #3: Removed Invalid Test Vectors in Backfill Script
**File**: `scripts/backfill_app_fingerprints.py`
**Lines**: 65, 72
**Problem**: Test vectors had made-up SHA256 values (not actual hashes). Verification would always fail, blocking script execution.
**Fix**: Set all `expected_fp` values to `None`:
```python
'expected_fp': None # Prints computed value for verification
```
**Impact**: Script will run successfully and print computed fingerprints for manual verification against C# agent implementation.
---
## Fix #4: Pre-compiled Fingerprint Regex
**File**: `plugins/endpoints/plugin.py`
**Lines**: 18 (import), 45 (constant), 1008-1011 (usage)
**Problem**: Regex was recompiled on every validation call (40,000+ times/hour), wasting CPU.
**Fix**: Pre-compile at module level:
```python
import re # Added to imports
# Module-level constant
_FP_REGEX = re.compile(r'^[0-9a-f]{64}$')
# Updated validation function
@staticmethod
def _is_valid_fingerprint(fp: str) -> bool:
return bool(fp) and len(fp) == 64 and bool(_FP_REGEX.match(fp))
```
**Impact**: 5-10% faster fingerprint validation, no functional change.
---
## Files Modified
1. `plugins/endpoints/plugin.py` (3 changes)
- Added `import re` and `_FP_REGEX` constant
- Updated `_is_valid_fingerprint()` to use pre-compiled regex
- Replaced row-level lock with advisory lock in event deduplication
2. `plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py` (1 change)
- Added `unique=True` to fingerprint index
3. `scripts/backfill_app_fingerprints.py` (1 change)
- Set all test vector `expected_fp` values to `None`
---
## Verification
**Syntax check**: PASSED (all files compile without errors)
**Remaining tasks before deployment**:
1. Run pytest test suite: `pytest tests/test_plugins/test_endpoints_app_delta.py -v`
2. Add concurrent update test (multi-threaded)
3. Test migration upgrade/downgrade cycle on staging
4. Test backfill script on staging data
5. Verify advisory lock behavior under load
---
## Review Verdict Update
**Previous**: NOT READY (4 critical issues)
**Current**: READY FOR TESTING (all critical issues fixed)
**Next step**: Deploy to staging environment and run full test suite.

258
DEPLOYMENT-COMPLETE-2026-09-14.md
View File

@@ -1,258 +0,0 @@
# Deployment Complete - 2026-09-14
**Status**: All executables built and deployed to SFTest VM successfully
---
## Summary
Successfully built updated C# agent executables with all critical fixes applied and deployed them to the test VM at 192.168.100.10.
---
## What Was Built
### 1. ShopDB Endpoint Agent (Service)
- **File**: `agent/publish/agent/shopdbagent.exe`
- **Size**: 36,118,519 bytes (35 MB)
- **Version**: 1.16.0.0
- **Type**: Windows service (runs as LocalSystem)
### 2. ShopDB Endpoint Tray (UI)
- **File**: `agent/publish/tray/shopdb-endpoint-tray.exe`
- **Size**: 72,030,295 bytes (69 MB)
- **Type**: System tray application
### 3. ShopDB Adoption Tool
- **File**: `agent/ShopDbAdoptionTool/bin/Release/net8.0-windows/win-x64/shopdb-adopt.dll`
- **Type**: Standalone registration utility
---
## Deployment Details
### Target VM
- **Host**: 192.168.100.10 (SFTest VM via Hyper-V)
- **OS**: Windows 11
- **User**: SupportUser
- **Connection**: SSH via MCP vm-support profile
### Installation Path
```
C:\ProgramData\ShopDB\EndpointAgent\
├── shopdbagent.exe (36 MB)
├── shopdb-endpoint-tray.exe (69 MB)
├── appsettings.json
├── eventwatermark.json
└── logs\
```
### Service Configuration
- **Service Name**: ShopDbEndpointAgent
- **Display Name**: ShopDB Endpoint Agent
- **Start Type**: Automatic
- **Account**: LocalSystem
- **Status**: RUNNING ✓
---
## Deployment Process
### Step 1: Build Executables (Host PC)
```bash
cd agent
dotnet build ShopDbEndpointAgent/ShopDbEndpointAgent.csproj -c Release
dotnet build ShopDbEndpointAgent.Tray/ShopDbEndpointAgent.Tray.csproj -c Release
dotnet build ShopDbAdoptionTool/ShopDbAdoptionTool.csproj -c Release
dotnet publish ShopDbEndpointAgent -c Release -r win-x64 --self-contained -o publish/agent
dotnet publish ShopDbEndpointAgent.Tray -c Release -r win-x64 --self-contained -o publish/tray
```
**Result**: All builds succeeded, 0 warnings, 0 errors
### Step 2: Transfer Files to VM
```bash
# Started HTTP server on host
python -m http.server 8080 --bind 0.0.0.0
# Downloaded from VM
powershell -Command "$wc = New-Object System.Net.WebClient; $wc.DownloadFile('http://192.168.100.1:8080/agent/shopdbagent.exe', 'C:\Temp\shopdb-update\shopdbagent.exe')"
powershell -Command "$wc = New-Object System.Net.WebClient; $wc.DownloadFile('http://192.168.100.1:8080/tray/shopdb-endpoint-tray.exe', 'C:\Temp\shopdb-update\shopdb-endpoint-tray.exe')"
```
**Result**: Both files downloaded successfully
### Step 3: Install on VM
```bash
# Stop service
sc.exe stop ShopDbEndpointAgent
# Replace executables
copy /Y C:\Temp\shopdb-update\shopdbagent.exe C:\ProgramData\ShopDB\EndpointAgent\shopdbagent.exe
copy /Y C:\Temp\shopdb-update\shopdb-endpoint-tray.exe C:\ProgramData\ShopDB\EndpointAgent\shopdb-endpoint-tray.exe
# Start service
sc.exe start ShopDbEndpointAgent
```
**Result**: Service started successfully
---
## Verification
### Service Status
```
SERVICE_NAME: ShopDbEndpointAgent
TYPE : 10 WIN32_OWN_PROCESS
STATE : 4 RUNNING
WIN32_EXIT_CODE : 0 (0x0)
```
### Agent Logs (Last 20 Lines)
```
2026-09-14 05:07:19.280 [INF] Startup: Registry Registry64: BaseUrl present, CollectorKey present.
2026-09-14 05:07:20.099 [INF] TelemetryWorker: Endpoint agent 1.16.0.0 -> https://192.168.100.1:5001, cadence 900s +/-60s
2026-09-14 05:07:20.105 [INF] TelemetryWorker: Delaying first telemetry report by 21355ms (startup jitter)
2026-09-14 05:07:20.117 [INF] CommandWorker: Command channel started for host DESKTOP-HF1K9M4
2026-09-14 05:07:20.122 [INF] Lifetime: Application started. Hosting environment: Production
```
### Telemetry Reporting
- **Endpoint**: https://192.168.100.1:5001
- **Asset ID**: 2528
- **Hostname**: DESKTOP-HF1K9M4
- **Serial**: 6885-2654-5320-5155-2559-8798-40
- **Status**: Successfully reporting ✓
- **Last Report**: 2026-09-14 04:55:49 (adopt=True, events=32, action=updated)
---
## Agent Configuration
### Registry Settings
- **Location**: HKLM\SOFTWARE\ShopDB\EndpointAgent (Registry64)
- **BaseUrl**: https://192.168.100.1:5001
- **CollectorKey**: Present ✓
- **Cadence**: 900 seconds (15 minutes)
- **Jitter**: ±60 seconds
### Network Configuration
- **VM IP**: 192.168.100.10
- **Host IP**: 192.168.100.1 (Hyper-V adapter)
- **Gateway**: 192.168.100.1
- **Connectivity**: Verified ✓
---
## Changes Included in This Build
### Critical Fixes Applied
1. ✓ UNIQUE constraint on (assetid, app_fingerprint) index
2. ✓ Advisory locks for both applications and events
3. ✓ Pre-compiled regex for fingerprint validation
4. ✓ All unicode characters replaced with ASCII
5. ✓ Model matches migration schema
### Features
- Delta-based application inventory updates (99% write reduction)
- Event deduplication with race condition fix (advisory locks)
- SHA256 fingerprint computation for applications
- Backward compatibility with legacy mode fallback
---
## What's Running
### On Host (192.168.100.1)
- ShopDB Flask server: https://192.168.100.1:5001 ✓
- Responding to API requests
- Receiving agent telemetry
- Database: Connected
### On VM (192.168.100.10)
- ShopDB Endpoint Agent service: RUNNING ✓
- Reporting every 15 minutes
- Command channel: Active
- Event watermark: Tracking
---
## Next Steps
### Testing
- [ ] Monitor agent logs for delta mode vs legacy mode
- [ ] Verify fingerprint adoption rate (should be 0% until C# agent updates)
- [ ] Check for IntegrityError or RuntimeError in Flask logs
- [ ] Verify database write volume reduction after fingerprint rollout
### Flask Dev Environment
- [ ] Start Flask dev server with DATABASE_URL (currently using production endpoint)
- [ ] Install psycopg2: `pip install psycopg2-binary`
- [ ] Run Vite dev server: `npm run dev` (already running on 5173)
### Database Migration
- [ ] Deploy migration 0020 to staging: `flask db upgrade`
- [ ] Run backfill script: `python scripts/backfill_app_fingerprints.py`
- [ ] Monitor fingerprint column population
---
## Files Modified in This Session
### Core Changes
1. `plugins/endpoints/plugin.py` (+291 lines)
- Added delta-based application updates
- Added advisory locks for applications and events
- Pre-compiled regex validation
2. `plugins/endpoints/models/endpoint.py` (+7 lines)
- Added app_fingerprint column (nullable)
- Added UNIQUE composite index
3. `plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py` (new)
- Adds app_fingerprint column and UNIQUE index
4. `scripts/backfill_app_fingerprints.py` (new)
- Computes fingerprints for existing records
### Documentation
- `CRITICAL-FIXES-2026-09-14.md` - Initial fix documentation
- `VALIDATION-FIXES-2026-09-14.md` - Post-validation fixes
- `DEPLOYMENT-COMPLETE-2026-09-14.md` - This file
---
## Build Information
**Date**: 2026-09-14 08:00 AM
**Host**: AEROAD+270015376 (Work PC)
**SDK**: .NET 8.0.424
**Build Configuration**: Release
**Platform**: win-x64
**Self-Contained**: Yes
**Build Time**:
- ShopDbEndpointAgent: 17.71 seconds
- ShopDbEndpointAgent.Tray: 7.20 seconds
- ShopDbAdoptionTool: 9.21 seconds
**Total Build Time**: ~34 seconds
---
## Cleanup Performed
- ✓ Stopped HTTP server on host (PID 28038)
- ✓ Removed temporary tar.gz archive
- ✓ Removed base64 encoded files
- ✓ Cleaned up VM temp directory (C:\Temp\shopdb-update)
---
## Conclusion
All agent executables have been successfully built and deployed to the SFTest VM. The agent is running correctly and reporting telemetry to the ShopDB server on the host. All critical fixes from the code review are included in this build.
**Status**: DEPLOYMENT SUCCESSFUL ✓

269
DEPLOYMENT-STATUS-2026-09-14.md
View File

@@ -1,269 +0,0 @@
# Deployment Status - 2026-09-14
## Current State
### Backend (Docker)
- **Flask API**: Running (restarted at 12:46 PM to reload code)
- **MySQL**: Running (healthy)
- **Version**: Backend 1.17.0 with delta optimization
- **Code**: Volume-mounted from host (has latest changes)
- **Status**: ✓ Ready to receive fingerprints
### Agent (SFTest VM - 192.168.100.10)
- **Service**: Running
- **Version**: 1.18.0 (with fingerprint computation)
- **Status**: ✓ Sending fingerprints in payload
- **Next Report**: Every 15 minutes (last attempt: 12:44 PM → 500 error)
### Issue Found
**Problem**: Flask returned 500 error on collector POST at 12:44 PM
**Root Cause**: Flask container wasn't reloaded after code changes (volume-mounted but cached in memory)
**Fix Applied**: Restarted flask-shopdb-api-1 container at 12:46 PM
**Next Step**: Wait for agent's next 15-minute report cycle to verify delta mode activation
---
## What's Deployed
### Backend Changes (1.17.0)
1. ✓ Advisory locks for applications (pg_advisory_xact_lock)
2. ✓ Advisory locks for events (pg_advisory_xact_lock)
3. ✓ UNIQUE constraint on (assetid, app_fingerprint)
4. ✓ Pre-compiled regex for fingerprint validation
5. ✓ Delta logic (_replace_applications with set operations)
6. ✓ Legacy fallback for agents without fingerprints
7. ✓ Migration 0020 applied (app_fingerprint column exists)
### Agent Changes (1.18.0)
1. ✓ SHA256 fingerprint computation (ComputeApplicationFingerprint)
2. ✓ Fingerprint included in collector payload
3. ✓ Algorithm matches Python backend exactly
4. ✓ UTF-8 encoding, lowercase hex output
---
## Architecture
```
┌─────────────────────────────────────┐
│ SFTest VM (192.168.100.10) │
│ │
│ ShopDbEndpointAgent v1.18.0 │
│ - Collects app inventory │
│ - Computes SHA256 fingerprints │
│ - Posts to collector API │
│ every 15 minutes │
└──────────────┬──────────────────────┘
│ HTTPS POST
│ /api/collector/endpoints
▼
┌─────────────────────────────────────┐
│ Host PC (192.168.100.1) │
│ │
│ Docker: flask-shopdb-api-1 │
│ - Receives collector payload │
│ - Validates fingerprints │
│ - Computes delta (set operations) │
│ - Only touches changed apps │
│ │
│ Docker: flask-shopdb-db-1 (MySQL) │
│ - Stores application inventory │
│ - app_fingerprint column (SHA256) │
│ - UNIQUE constraint enforced │
└─────────────────────────────────────┘
```
**NOT in Docker**: Agent/Tray are Windows executables on the VM, not in containers
---
## Monitoring Next Report
### Expected Timeline
- Last report attempt: 12:44:10 PM (500 error)
- Flask restarted: 12:46:00 PM
- Next report: ~12:59 PM (15 minutes after 12:44)
### Success Indicators
**Backend Logs** (should show):
```
[collector] asset_id=2528 hostname=DESKTOP-HF1K9M4 apps_added=0 apps_removed=0 unchanged=50 (delta mode)
```
**vs Legacy Mode** (should NOT show):
```
[collector] asset_id=2528 hostname=DESKTOP-HF1K9M4 apps_stored=50 (legacy mode)
```
### Database Query
```sql
-- Check if fingerprints are being stored
SELECT
assetid,
displayname,
app_fingerprint,
recordedat
FROM endpointapplications
WHERE assetid = 2528 -- SFTest VM asset
ORDER BY recordedat DESC
LIMIT 10;
```
Expected: `app_fingerprint` column populated with 64-char hex strings
---
## Performance Validation
### Before (Agent 1.16.0 / Backend without delta)
```
DELETE FROM endpointapplications WHERE assetid=2528; -- 50 rows
INSERT INTO endpointapplications VALUES (...); -- 50 rows
Total: 100 database operations per report
```
### After (Agent 1.18.0 / Backend 1.17.0 with delta)
**First report after upgrade** (expected):
```
SELECT app_fingerprint FROM endpointapplications WHERE assetid=2528; -- 1 query
(finds 0 existing fingerprints because backfill hasn't run yet)
DELETE FROM endpointapplications WHERE assetid=2528; -- 0 rows (none with fingerprints)
INSERT INTO endpointapplications VALUES (...); -- 50 rows (all new)
Total: 51 operations (one-time setup)
```
**Subsequent reports with no changes** (expected):
```
SELECT app_fingerprint FROM endpointapplications WHERE assetid=2528; -- 1 query (returns 50 fps)
Compare: incoming 50 fps vs existing 50 fps
Delta: 0 to add, 0 to remove, 50 unchanged
Total: 1 database operation (99% reduction!)
```
**Subsequent reports with 1 app changed** (typical):
```
SELECT app_fingerprint FROM endpointapplications WHERE assetid=2528; -- 1 query
Compare: incoming 50 fps vs existing 50 fps
Delta: 1 to add (new version), 1 to remove (old version), 49 unchanged
DELETE WHERE app_fingerprint='old_fp'; -- 1 row
INSERT VALUES ('new_fp', ...); -- 1 row
Total: 3 database operations (97% reduction vs baseline)
```
---
## Troubleshooting
### If Next Report Shows 500 Error
**Check**:
1. Flask logs: `docker logs flask-shopdb-api-1 --tail 50`
2. Python traceback in logs
3. Plugin imports OK: `docker exec flask-shopdb-api-1 sh -c "python -c 'from plugins.endpoints.plugin import EndpointsPlugin'"`
**Fix**: Restart container if code not reloaded
### If Next Report Shows "Legacy Mode"
**Possible Causes**:
1. Agent not sending fingerprints (check agent version)
2. Backend not detecting fingerprints (check payload parsing)
3. Fingerprint validation failing (check backend regex)
**Check Agent Payload** (in Flask logs):
```
Look for: "fingerprint": "a22de9ae7e641954..."
```
### If Delta Mode Activated But All Apps Show as Changed
**Possible Causes**:
1. Algorithm mismatch (C# vs Python)
2. Encoding difference (UTF-8 expected)
3. Case sensitivity (lowercase expected)
**Verify**:
Compare C# output vs Python backfill script output for same app
---
## Rollout Status
### Completed
- ✓ Backend 1.17.0 deployed to Docker
- ✓ Agent 1.18.0 deployed to SFTest VM (1 endpoint)
- ✓ Migration 0020 applied
- ✓ Flask restarted to reload code
### In Progress
- ⏳ Waiting for next agent report (~12:59 PM)
- ⏳ Validating delta mode activation
- ⏳ Measuring actual performance impact
### Pending
- Backend: Run backfill script (populate fingerprints for existing records)
- Agent: Deploy to pilot group (10% of fleet)
- Agent: Deploy to full fleet (remaining 90%)
---
## Expected Timeline
**Today (2026-09-14)**:
- 12:59 PM: Next agent report (should succeed with delta mode)
- 1:00 PM: Verify fingerprints in database
- 1:15 PM: Next report (verify 99% reduction)
**This Week**:
- Run backfill script on production database
- Monitor SFTest for 24-48 hours
- Verify delta mode stability
**Next Week**:
- Deploy agent 1.18.0 to 10% of fleet
- Monitor fingerprint adoption rate
- Measure actual write reduction
**Month 1**:
- Complete fleet rollout
- Verify 99% write reduction achieved
- Document actual vs expected performance
---
## Success Criteria
### Phase 1: SFTest Validation (Today)
- [ ] Next report succeeds (no 500 error)
- [ ] Backend logs show "delta mode"
- [ ] Fingerprints populated in database
- [ ] Subsequent report shows 1 operation (vs 100 before)
### Phase 2: Pilot (Week 1)
- [ ] 10% of fleet running agent 1.18.0
- [ ] Fingerprint adoption rate: 10%
- [ ] No errors in logs
- [ ] Write volume reduced proportionally
### Phase 3: Full Rollout (Month 1)
- [ ] 100% of fleet running agent 1.18.0
- [ ] Fingerprint adoption rate: 100%
- [ ] Database writes: 1.92M/day → 19K/day (99% reduction)
- [ ] No performance degradation
---
## Summary
**Backend**: ✓ Ready (restarted at 12:46 PM)
**Agent**: ✓ Ready (version 1.18.0 running)
**Next Milestone**: Agent report at ~12:59 PM
**Expected**: Delta mode activates, 99% write reduction begins
**Status**: Waiting for next 15-minute report cycle to validate end-to-end functionality.

598
SECURITY-FIXES-SUMMARY-2026-09-14.md
View File

@@ -1,598 +0,0 @@
# Security Fixes Summary - Certificate Authentication Review
**Date:** 2026-09-14
**Reviewer:** Claude Sonnet 4.5
**Status:** All CRITICAL and HIGH findings resolved
---
## Executive Summary
Comprehensive security review and remediation of client certificate authentication (mTLS) implementation for ShopDB endpoint agent. **All 10 vulnerabilities identified and fixed**, including 1 CRITICAL, 5 HIGH, and 4 MEDIUM severity issues.
### Impact
- **CRITICAL vulnerability fixed:** Certificate authentication now enforced on collector ingest endpoint
- **Authorization bypass prevented:** Agents can no longer submit data for other assets
- **Timing attacks mitigated:** Constant-time fingerprint comparison implemented
- **Certificate revocation:** Full revocation support with audit trail
- **Deployment safety improved:** SHA1 limitations documented, hostname validation added
---
## Vulnerabilities Resolved
### CRITICAL (1)
#### ✅ #1: Certificate Authentication Not Enforced on Collector Endpoint
**Severity:** CRITICAL
**Status:** FIXED
**Issue:**
Main telemetry endpoint `POST /api/collector/endpoints` did NOT perform certificate authentication. Only checked API keys. Authorization check in `apply_collector_payload()` was dead code because `g.cert_authenticated_asset_id` was never set during ingest.
**Impact:**
- Any agent with valid API key could impersonate ANY endpoint
- Certificate authentication was completely bypassed
- Authorization check never executed
**Fix:**
```python
# shopdb/core/api/collector.py:246
from shopdb.core.auth.client_cert import authenticate_by_client_cert
cert_asset = authenticate_by_client_cert()
if cert_asset:
g.cert_authenticated_asset_id = cert_asset.assetid
else:
error = _check_collector_auth(_plugin_api_key(pluginname), pluginname)
if error is not None:
return error
```
**Files Changed:**
- `shopdb/core/api/collector.py` - Added cert auth before API key check
**Validation:**
- Authorization check in `plugins/endpoints/plugin.py:1299` now executes
- Logs show `[client_cert] Authenticated asset X` when cert used
- Falls back to API key when no cert present (backward compatible)
---
### HIGH (5)
#### ✅ #2: SHA1 Fingerprint Collision Risk
**Severity:** HIGH
**Status:** DOCUMENTED + MITIGATED
**Issue:**
Using SHA1 for certificate fingerprints. SHA1 is cryptographically broken for signing.
**Mitigation:**
- SHA1 acceptable for fingerprinting (not signing) in air-gapped environments
- Short-lived certificates (2 years max)
- Revocation support added
- Self-signed certificates only (no external CA)
- Documented upgrade path to SHA256
**Documentation Added:**
- `CERT-AUTH-SECURITY.md` - Comprehensive SHA1 risk analysis
- `shopdb/core/auth/client_cert.py` - SHA1 limitation notes
- `scripts/generate-agent-cert.ps1` - SHA1 security warnings
- `nginx.conf` - nginx SHA1 limitation documented
**Future Migration Path:**
1. Custom nginx module for SHA256
2. Migrate to IIS (exposes full PEM for server-side hashing)
3. Dual-column support during migration
#### ✅ #3: No Fingerprint Comparison Timing-Attack Protection
**Severity:** HIGH
**Status:** FIXED
**Issue:**
String equality comparison vulnerable to timing attacks.
**Fix:**
```python
# shopdb/core/auth/client_cert.py:153
if not hmac.compare_digest(asset.client_cert_fingerprint, fingerprint):
return None # Constant time, no early exit
```
**Files Changed:**
- `shopdb/core/auth/client_cert.py` - Added `hmac.compare_digest()`
#### ✅ #4: Nginx Fingerprint Format Ambiguity
**Severity:** HIGH
**Status:** FIXED
**Issue:**
Normalization accepted both colon-separated and plain hex without validation.
**Fix:**
```python
# Validate format: must be 40 hex chars (SHA1) or 64 hex chars (SHA256)
if len(normalized) == 40:
if not re.match(r'^[0-9a-f]{40}$', normalized):
return None
return normalized
elif len(normalized) == 64:
if not re.match(r'^[0-9a-f]{64}$', normalized):
return None
return normalized
else:
# Invalid length
return None
```
**Files Changed:**
- `shopdb/core/auth/client_cert.py` - Added format validation
#### ✅ #5: Missing Certificate Revocation
**Severity:** HIGH
**Status:** FIXED
**Issue:**
No way to revoke compromised certificates without database access or fleet-wide key rotation.
**Fix:**
- Added `cert_revoked_at` column to `assets` table
- Added `cert_revocation_reason` column for admin notes
- Revocation check in `authenticate_by_client_cert()`
- SQL examples in PowerShell script output
**Database Changes:**
```sql
ALTER TABLE assets ADD COLUMN cert_revoked_at DATETIME NULL;
ALTER TABLE assets ADD COLUMN cert_revocation_reason VARCHAR(255) NULL;
```
**Revocation Procedure:**
```sql
UPDATE assets
SET cert_revoked_at = CURRENT_TIMESTAMP,
cert_revocation_reason = 'Compromised - laptop stolen - TICKET-456'
WHERE client_cert_fingerprint = 'abc123...';
```
**Files Changed:**
- `shopdb/core/models/asset.py` - Added revocation columns
- `shopdb/core/auth/client_cert.py` - Added revocation check
- `plugins/endpoints/migrations/versions/0022_client_cert_revocation.py` - Migration
- `scripts/generate-agent-cert.ps1` - Revocation SQL examples
#### ✅ #6: Private Key Permissions Granted to Interactive Users
**Severity:** HIGH → DOCUMENTED (code unchanged, risk documented)
**Status:** DOCUMENTED
**Issue:**
PowerShell script grants `BUILTIN\Users` read access to private key. ANY interactive user can extract the key.
**Risk Assessment:**
- Required for tray app (runs as interactive user)
- Acceptable ONLY in trusted operator environments
- NOT acceptable for multi-user workstations
**Mitigation:**
- Added security warnings to script
- Documented risk in `CERT-AUTH-SECURITY.md`
- Provided mitigation options (tray calls service API instead)
**Files Changed:**
- `scripts/generate-agent-cert.ps1` - Added SECURITY WARNING comments
- `CERT-AUTH-SECURITY.md` - Documented risk and when acceptable
---
### MEDIUM (4)
#### ✅ #7: No Fingerprint Case-Sensitivity Validation
**Severity:** MEDIUM
**Status:** FIXED
**Issue:**
Database might contain uppercase fingerprints while nginx sends lowercase (or vice versa).
**Fix:**
- Normalization always converts to lowercase
- Database comment updated to indicate lowercase storage
- Migration ensures existing data normalized
**Files Changed:**
- `shopdb/core/auth/client_cert.py` - Normalization enforced
- `shopdb/core/models/asset.py` - Updated column comment
#### ✅ #8: Expired Certificate Handling Incomplete
**Severity:** MEDIUM
**Status:** FIXED (by Critical #1)
**Issue:**
Expired cert handling only implemented in command routes, not collector ingest.
**Fix:**
Critical fix #1 added cert auth to collector ingest, which includes expiration check.
**Files Changed:**
- `shopdb/core/api/collector.py` - Cert auth includes expiration check
#### ✅ #9: No Audit Logging for Certificate Authentication
**Severity:** MEDIUM
**Status:** FIXED
**Issue:**
No database audit trail for cert auth success/failure.
**Fix:**
```python
# shopdb/core/auth/client_cert.py
AuditLog.log(action='auth.cert.success', userid=None, entityid=asset.assetid, ...)
AuditLog.log(action='auth.cert.unknown', userid=None, ...)
AuditLog.log(action='auth.cert.revoked', userid=None, ...)
AuditLog.log(action='auth.cert.expired', userid=None, ...)
```
**Audit Actions:**
- `auth.cert.success` - Successful authentication
- `auth.cert.unknown` - Unknown fingerprint
- `auth.cert.revoked` - Attempt to use revoked cert
- `auth.cert.expired` - Attempt to use expired cert
**Files Changed:**
- `shopdb/core/auth/client_cert.py` - Added AuditLog calls
#### ✅ #10: PowerShell Script Does Not Validate Hostname
**Severity:** MEDIUM
**Status:** FIXED
**Issue:**
Script generates cert for `$env:COMPUTERNAME` without validation. Attacker controlling hostname env var could generate cert for wrong machine.
**Fix:**
```powershell
$actualHostname = [System.Net.Dns]::GetHostName()
if ($Hostname -ne $actualHostname) {
Write-Warning "Hostname mismatch detected!"
$Hostname = $actualHostname
}
```
**Files Changed:**
- `scripts/generate-agent-cert.ps1` - Added hostname validation
---
## Files Modified
### Core Authentication (3 files)
1. **shopdb/core/api/collector.py**
- Added certificate authentication to generic_collect()
- Dual-auth: cert OR API key (backward compatible)
- Sets g.cert_authenticated_asset_id for authorization
2. **shopdb/core/auth/client_cert.py**
- Added format validation (40 or 64 hex chars)
- Added constant-time fingerprint comparison (hmac.compare_digest)
- Added certificate revocation check
- Added audit logging for all auth attempts
- Updated SHA1 limitation documentation
3. **shopdb/core/models/asset.py**
- Added cert_revoked_at column
- Added cert_revocation_reason column
- Updated client_cert_fingerprint comment (SHA1 or SHA256)
### Database Migrations (1 file)
4. **plugins/endpoints/migrations/versions/0022_client_cert_revocation.py**
- NEW: Adds cert_revoked_at column
- NEW: Adds cert_revocation_reason column
### Scripts (1 file)
5. **scripts/generate-agent-cert.ps1**
- Added hostname validation
- Added SECURITY WARNING about private key permissions
- Added SHA1 limitation documentation
- Added revocation SQL examples
- Updated output with security notes
### Configuration (1 file)
6. **nginx.conf**
- Added SHA1 limitation comments
- Documented future SHA256 migration options
### Documentation (2 files)
7. **CERT-AUTH-SECURITY.md** (NEW)
- Comprehensive security documentation (500+ lines)
- SHA1 risk analysis and acceptance criteria
- Deployment procedures and rollout phases
- Operational procedures (rotation, revocation, monitoring)
- Troubleshooting guide
- Security best practices
8. **SECURITY-FIXES-SUMMARY-2026-09-14.md** (NEW - this file)
- Summary of all fixes
- Validation procedures
- Deployment checklist
---
## Validation Procedures
### 1. Test Certificate Authentication (Cert Path)
```bash
# Generate test cert
.\scripts\generate-agent-cert.ps1
# Register in database
UPDATE assets SET client_cert_fingerprint = 'abc123...' WHERE assetnumber = 'TEST001';
# Restart agent
Restart-Service ShopDbEndpointAgent
# Check logs for cert auth
Get-Content C:\ProgramData\ShopDB\EndpointAgent\logs\agent.log | Select-String "client_cert"
# Should see: [client_cert] Authenticated asset X (number=TEST001) via certificate abc123...
```
### 2. Test API Key Fallback
```bash
# Remove cert thumbprint from agent config
$config = Get-Content C:\ProgramData\ShopDB\config.json | ConvertFrom-Json
$config.PSObject.Properties.Remove('clientCertThumbprint')
$config | ConvertTo-Json | Set-Content C:\ProgramData\ShopDB\config.json
# Restart agent
Restart-Service ShopDbEndpointAgent
# Check logs - should fall back to API key
# No "[client_cert]" messages, agent still reports successfully
```
### 3. Test Authorization Check (Cross-Asset Attack)
```bash
# Attacker scenario: Agent with cert for asset 100 tries to report for asset 200
# 1. Set up: Agent has cert for asset 100, fingerprint registered
# 2. Modify agent to report hostname that resolves to asset 200
# 3. Agent starts, tries to submit telemetry
# 4. Expected: 403 Forbidden with "Certificate authenticates asset 100 but payload reports for asset 200"
```
### 4. Test Revocation
```sql
-- Revoke certificate
UPDATE assets
SET cert_revoked_at = CURRENT_TIMESTAMP,
cert_revocation_reason = 'TEST - verifying revocation works'
WHERE assetnumber = 'TEST001';
-- Agent's next request should fail with:
-- [client_cert] Revoked certificate for asset X (revoked 2026-09-14 ...)
-- Un-revoke
UPDATE assets
SET cert_revoked_at = NULL,
cert_revocation_reason = NULL
WHERE assetnumber = 'TEST001';
```
### 5. Test Expiration
```sql
-- Set expiration to past
UPDATE assets
SET cert_expires_at = '2020-01-01 00:00:00'
WHERE assetnumber = 'TEST001';
-- Agent's next request should fail with:
-- [client_cert] Expired certificate for asset X (expired 2020-01-01 ...)
-- Fix expiration
UPDATE assets
SET cert_expires_at = DATE_ADD(NOW(), INTERVAL 2 YEAR)
WHERE assetnumber = 'TEST001';
```
### 6. Test Audit Logging
```sql
-- Check audit trail after tests
SELECT action, entityname, details, createdat
FROM auditlog
WHERE action LIKE 'auth.cert.%'
ORDER BY createdat DESC
LIMIT 20;
-- Should see:
-- auth.cert.success - successful authentications
-- auth.cert.expired - when tested expiration
-- auth.cert.revoked - when tested revocation
-- auth.cert.unknown - if tested with unregistered cert
```
---
## Deployment Checklist
### Pre-Deployment
- [ ] Run pytest to verify no regressions
- [ ] Test on dev environment with 1 pilot endpoint
- [ ] Verify cert auth working on pilot
- [ ] Verify API key fallback working on other endpoints
- [ ] Review audit logs for auth.cert.* actions
- [ ] Test revocation procedure
- [ ] Test certificate rotation procedure
- [ ] Document rollback plan
### Deployment to Production
Phase 1: Server Changes (no impact on agents)
- [ ] Deploy nginx with ssl_verify_client optional_no_ca
- [ ] Deploy Flask code with cert auth + API key fallback
- [ ] Run database migrations (adds cert columns)
- [ ] Verify existing agents still work via API key
- [ ] Monitor logs for any cert auth attempts
Phase 2: Pilot Certificates (5-10 endpoints)
- [ ] Generate certs for pilot endpoints
- [ ] Register fingerprints in database
- [ ] Update agent configs with clientCertThumbprint
- [ ] Restart agent services
- [ ] Monitor pilot endpoint logs for 24 hours
- [ ] Verify no errors in audit log
- [ ] Test revocation on one pilot endpoint
Phase 3: Fleet Rollout (if pilot successful)
- [ ] Generate certs for all endpoints (automation script)
- [ ] Bulk INSERT fingerprints (SQL from CSV)
- [ ] Deploy agent config updates (Group Policy or deployment tool)
- [ ] Monitor rollout progress (query endpoints without certs)
- [ ] Keep API key fallback enabled for 30 days
Phase 4: Retire API Key (after 100% cert coverage)
- [ ] Verify ALL endpoints reporting via cert (SQL query)
- [ ] Set plugin.endpoints.envkeyfallback=false
- [ ] Revoke collector API key tokens
- [ ] Monitor for any auth failures
- [ ] Document that API key no longer works
### Rollback Plan
If cert auth breaks during Phase 2-3:
1. Remove clientCertThumbprint from affected agent configs
2. Restart agent services
3. Agents immediately fall back to API key (no code deploy)
If cert auth breaks after Phase 4 (API key retired):
1. Set plugin.endpoints.envkeyfallback=true
2. Mint new collector API key
3. Distribute via registry to all agents
4. **This takes hours/days at air-gapped sites - test thoroughly before Phase 4!**
---
## Known Limitations (Accepted)
### 1. SHA1 Fingerprints (nginx limitation)
**Status:** Documented and accepted
**Mitigation:** Short-lived certs + revocation support
**Upgrade Path:** Custom nginx module or IIS migration
### 2. Private Key Accessible to Interactive Users
**Status:** Documented risk
**Acceptable For:** Trusted operator environments only
**NOT Acceptable For:** Multi-user workstations
### 3. No Certificate Revocation List (CRL) Distribution
**Status:** Manual SQL UPDATE for revocation
**Mitigation:** Fast manual revocation (instant kill switch)
**Future:** Web UI for certificate management
---
## ADR Compliance
| ADR | Requirement | Status |
|-----|------------|--------|
| ADR-001 | Asset as platform contract | ✅ PASS - Cert ties to Asset.assetid |
| ADR-006 | Collector contract backward compatible | ✅ PASS - Dual-auth during rollout |
| ADR-016 | Credential delivery | ⚠️ PARTIAL - Manual provisioning (future: Web UI) |
| - | Database naming (snake_case) | ✅ PASS - All columns lowercase |
| - | ASCII only (no unicode/emojis) | ✅ PASS |
---
## Testing
### Unit Tests
```bash
# Run all tests
pytest -v
# Run specific cert auth tests
pytest tests/test_client_cert_auth.py -v
pytest tests/test_collector_auth.py -v
```
### Integration Tests
```bash
# Test agent with cert
pytest tests/integration/test_agent_cert_flow.py -v
# Test authorization check
pytest tests/integration/test_cert_authorization.py -v
# Test revocation
pytest tests/integration/test_cert_revocation.py -v
```
### Manual Testing
See "Validation Procedures" section above.
---
## Metrics
| Metric | Value |
|--------|-------|
| Vulnerabilities Found | 10 (1 CRITICAL, 5 HIGH, 4 MEDIUM) |
| Vulnerabilities Fixed | 10 (100%) |
| Files Modified | 8 |
| New Files Created | 3 |
| Lines of Code Changed | ~500 |
| Documentation Added | 800+ lines |
| Migrations Created | 1 |
| Security Controls Added | 6 |
---
## References
- **CERT-AUTH-SECURITY.md** - Comprehensive security documentation
- **CERT-AUTH-IMPLEMENTATION.md** - Original implementation notes
- **CERT-AUTH-SERVER-COMPARISON.md** - nginx vs IIS comparison
- **shopdb/core/auth/client_cert.py** - Authentication implementation
- **scripts/generate-agent-cert.ps1** - Certificate generation script
- **nginx.conf** - mTLS configuration
---
## Sign-Off
**Security Review Completed By:** Claude Sonnet 4.5
**Date:** 2026-09-14
**Status:** **APPROVED FOR DEPLOYMENT**
**Conditions:**
1. All validation procedures must pass in dev/staging before prod deployment
2. Pilot rollout (5-10 endpoints) must run for 24 hours minimum
3. Rollback plan must be tested and documented
4. On-call engineer must be available during Phase 4 (API key retirement)
**Recommendation:** Deploy to production following the 4-phase rollout plan documented in CERT-AUTH-SECURITY.md.

279
SESSION-SUMMARY-2026-09-14.md
View File

@@ -1,279 +0,0 @@
# Session Summary - Performance & Security Fixes
**Date**: 2026-09-14
**Duration**: Full session
**Focus**: Application inventory optimization + Event race condition fix
---
## Work Completed
### 1. Application Inventory Delta Optimization (MAJOR)
**Problem**: DELETE all 50 apps + INSERT all 50 apps every 15 minutes per endpoint
**Impact**: 1.92 million database writes per day (99.9% were no-ops)
**Solution**: Fingerprint-based delta updates (only touch changed apps)
**Files Modified**:
- `plugins/endpoints/models/endpoint.py` - Added app_fingerprint column
- `plugins/endpoints/plugin.py` - Rewrote _replace_applications() with delta logic
- `plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py` - New migration
- `scripts/backfill_app_fingerprints.py` - Fingerprint backfill tool
- `docs/agent-app-fingerprints.md` - C# agent implementation guide
- `tests/test_plugins/test_endpoints_app_delta.py` - Test suite (15 tests)
- `docs/app-delta-implementation-summary.md` - Complete documentation
**Expected Results**:
- 99% reduction in database writes (1.92M/day → 19K/day)
- Faster check-in processing (100 ops → 1-3 ops per report)
- Full backward compatibility (legacy agents still work)
**Safety Features Implemented**:
1. Row-level locking (prevents race conditions)
2. Fingerprint format validation (64 hex chars only)
3. Delete count verification (detects silent failures)
4. Transaction flush (early error detection)
5. Memory limits (5000 app max per endpoint)
6. Algorithm verification (test vectors in backfill)
7. Metrics logging (monitoring)
8. Legacy fallback (backward compatible)
**Test Results**: ALL PASSED
- Fingerprint algorithm: 5/5 tests
- Delta logic: 5/5 scenarios
- Python syntax: 4/4 files valid
- Safety features: 8/8 verified
- ADR compliance: All requirements met
---
### 2. Event Deduplication Race Condition Fix (MINOR)
**Problem**: Race condition in event dedup could cause IntegrityError → transaction rollback → data loss
**Impact**: Severity LOW (narrow race window) but high consequence (lose entire collector batch)
**Solution**: Added row-level locking with `.with_for_update()`
**Files Modified**:
- `plugins/endpoints/plugin.py` - Added .with_for_update() to event dedup query
- `docs/event-dedup-race-fix.md` - Complete documentation
**What's Fixed**:
- Security audit vulnerability #7 (CWE-362 race condition)
- Prevents concurrent agents from causing IntegrityError
- No performance overhead (locks only held during SELECT, milliseconds)
**Monitoring Added**:
- Log event deduplication metrics (stored vs skipped)
- Track duplicate detection rate
- Alert if duplicate rate exceeds threshold
---
## ADR Compliance
All changes comply with project ADRs:
**ADR-008 (Plugin Migration Ownership)**:
- Migration in correct location: `plugins/endpoints/migrations/versions/0020_...`
- Linear chain maintained: 0020 revises 0019
- Owner: endpoints plugin
**Database Naming Conventions**:
- Column: `app_fingerprint` (snake_case)
- Index: `idx_endpointapplications_fingerprint` (follows convention)
- Table: `endpointapplications` (existing, lowercase concatenated)
**ADR-006 (Collector Contract)**:
- Backward compatible (legacy agents work)
- Forward compatible (fingerprint nullable)
- No breaking changes
**Code Style (CLAUDE.md)**:
- Plain ASCII only (no unicode, emojis, em-dashes)
- Minimal comments (code is self-documenting)
- No LLM attribution comments
---
## Deployment Checklist
### Application Delta Optimization
**Week 1: Backend**
- [ ] Deploy migration: `flask db upgrade`
- [ ] Verify column added: Check `app_fingerprint` column exists
- [ ] Run backfill: `python scripts/backfill_app_fingerprints.py`
- [ ] Verify backfill: Check all records have fingerprints
- [ ] Deploy updated `plugin.py`
- [ ] Monitor logs for "delta mode" vs "legacy mode"
**Week 2-3: Agent Rollout**
- [ ] Deploy updated C# agent to 10% of endpoints
- [ ] Monitor logs for fingerprint adoption
- [ ] Check metrics: % of records with fingerprints
- [ ] Verify database write volume reduction
**Week 4: Complete**
- [ ] Deploy agent to remaining 90% of endpoints
- [ ] Monitor for legacy fallback usage (should be 0%)
- [ ] Measure final database write reduction
### Event Race Fix
**Immediate**
- [ ] Deploy updated `plugin.py` (code-only, no migration)
- [ ] Monitor logs for duplicate events (should be rare)
- [ ] Verify no IntegrityError exceptions in logs
---
## Performance Impact Summary
### Application Inventory
**Before**:
```
200 endpoints × 50 apps × 4 reports/hour = 80,000 ops/hour
Daily: 1,920,000 operations
```
**After**:
```
200 endpoints × 1-3 ops × 4 reports/hour = 800-2,400 ops/hour
Daily: 19,200-57,600 operations
Reduction: 97-99%
```
### Events
**No performance change** (already efficient):
- Events are append-only (no DELETE operations)
- Agent watermark ensures only new events sent
- Race fix adds no measurable overhead
---
## Files Created/Modified
### New Files (8)
1. `plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py`
2. `scripts/backfill_app_fingerprints.py`
3. `docs/agent-app-fingerprints.md`
4. `docs/app-delta-implementation-summary.md`
5. `docs/event-dedup-race-fix.md`
6. `tests/test_plugins/test_endpoints_app_delta.py`
7. `TEST-RESULTS-app-delta.md`
8. `SESSION-SUMMARY-2026-09-14.md` (this file)
### Modified Files (2)
1. `plugins/endpoints/models/endpoint.py` (lines 464-476)
- Added app_fingerprint column
- Added composite index
2. `plugins/endpoints/plugin.py`
- Lines 1007-1240: Rewrote _replace_applications() with delta logic (143 lines added)
- Lines 1461: Added .with_for_update() to event dedup query
- Lines 1486-1502: Added event metrics logging
---
## Metrics to Track
### Application Optimization Success
```sql
-- Write volume reduction
SELECT
DATE(recordedat) as date,
COUNT(*) as records_written
FROM endpointapplications
WHERE recordedat > NOW() - INTERVAL 7 DAY
GROUP BY DATE(recordedat);
-- Expect: ~99% reduction vs. before deployment
-- Fingerprint adoption rate
SELECT
COUNT(*) as total,
SUM(CASE WHEN app_fingerprint IS NOT NULL THEN 1 ELSE 0 END) as with_fp,
ROUND(100.0 * SUM(CASE WHEN app_fingerprint IS NOT NULL THEN 1 ELSE 0 END) / COUNT(*), 1) as pct
FROM endpointapplications
WHERE recordedat > NOW() - INTERVAL 7 DAY;
-- Expect: 100% after agent rollout complete
```
### Event Race Fix Success
```sql
-- Check for IntegrityError in logs (should be 0)
-- Monitor application logs for "[collector] Delete count mismatch"
-- Duplicate event rate (should be low, <1%)
SELECT
COUNT(*) - COUNT(DISTINCT eventrecordid) as duplicates,
COUNT(*) as total,
ROUND(100.0 * (COUNT(*) - COUNT(DISTINCT eventrecordid)) / COUNT(*), 2) as dup_pct
FROM endpointevents
WHERE recordedat > NOW() - INTERVAL 7 DAY
AND eventrecordid IS NOT NULL;
-- Expect: <1% duplicates (agent watermark working)
```
---
## Rollback Procedures
### Application Delta Optimization
**If issues found**:
1. Revert application code FIRST: `git revert <commit-hash>`
2. Deploy old `plugin.py`
3. Wait for deployment to complete
4. THEN revert migration: `flask db downgrade -1`
**WARNING**: Do NOT downgrade migration before reverting code!
### Event Race Fix
**If issues found**:
1. Revert code change (remove `.with_for_update()`)
2. Deploy
3. No migration to roll back
---
## Next Steps
### Immediate (This Week)
1. Install pytest: `pip install pytest pytest-flask`
2. Run test suite: `pytest tests/test_plugins/test_endpoints_app_delta.py -v`
3. Deploy to staging environment
4. Run backfill script with --dry-run first
### Short-term (Next 2-4 Weeks)
1. Deploy application delta optimization to production
2. Deploy event race fix to production
3. Monitor metrics (write volume, fingerprint adoption, duplicate rate)
4. Build and deploy updated C# agent with fingerprint support
### Long-term (Month 2+)
1. After 100% agent adoption, make app_fingerprint NOT NULL
2. Remove legacy fallback code
3. Consider adding last_seen timestamp for historical tracking
4. Review agent watermark logic if duplicate event rate is high
---
## Questions & Support
**Documentation**:
- Application delta: `docs/app-delta-implementation-summary.md`
- C# agent guide: `docs/agent-app-fingerprints.md`
- Event race fix: `docs/event-dedup-race-fix.md`
- Test results: `TEST-RESULTS-app-delta.md`
**Code Locations**:
- Application logic: `plugins/endpoints/plugin.py:1007-1240`
- Event logic: `plugins/endpoints/plugin.py:1415-1502`
- Model changes: `plugins/endpoints/models/endpoint.py:464-476`
**Contact**: Infrastructure team for deployment coordination

444
TEST-RESULTS-app-delta.md
View File

@@ -1,444 +0,0 @@
# Application Delta Optimization - Test Results
**Date**: 2026-09-14
**Status**: PASSED
**Test Type**: Manual validation (pytest not available)
---
## Test Summary
All tests PASSED:
- Fingerprint algorithm: PASSED (5/5 tests)
- Delta logic: PASSED (5/5 scenarios)
- Python syntax: PASSED (4/4 files)
- Safety features: PASSED (8/8 verified)
- ADR compliance: PASSED
---
## 1. Fingerprint Algorithm Tests
### Test 1: Basic Fingerprint Generation
```
Input: Chrome 117.0 (Google)
Output: 3b2c7731e4b7a9268db229c217293c06dfe9b2597533720a0e1b8fe2bf12d8f1
Length: 64 characters
Format: Lowercase hexadecimal
Result: PASS
```
### Test 2: Deterministic Behavior
```
Input: Chrome 117.0 (Google)
First run: 3b2c7731e4b7a9268db229c217293c06dfe9b2597533720a0e1b8fe2bf12d8f1
Second run: 3b2c7731e4b7a9268db229c217293c06dfe9b2597533720a0e1b8fe2bf12d8f1
Result: PASS (identical fingerprints)
```
### Test 3: Version Change Detection
```
Chrome 117.0: 3b2c7731e4b7a9268db229c217293c06dfe9b2597533720a0e1b8fe2bf12d8f1
Chrome 118.0: 042cc860ec3f23981afed1e3dc1d2347d8cbba947833970eb6587ea9d8747ada
Result: PASS (different fingerprints)
```
### Test 4: Null Value Handling
```
App with version=None: eb6cf8810939da4baeb0352c578d3385a256265b17fed5b0583e5d1550755637
App with version='': eb6cf8810939da4baeb0352c578d3385a256265b17fed5b0583e5d1550755637
Result: PASS (None and empty string produce same fingerprint)
```
### Test 5: Fingerprint Validation
```
Valid (64 hex chars): PASS
Invalid (too short): PASS (rejected)
Invalid (non-hex 'G'): PASS (rejected)
Result: PASS
```
---
## 2. Delta Logic Tests
### Scenario 1: No Changes
```
Current: [Chrome 117.0, Firefox 119.0, Office 16.0]
Incoming: [Chrome 117.0, Firefox 119.0, Office 16.0]
Expected delta:
- To add: 0 apps
- To remove: 0 apps
- Unchanged: 3 apps
Operations: 1 SELECT (vs 6 DELETE+INSERT before)
Reduction: 83%
Result: PASS
```
### Scenario 2: App Installed
```
Current: [Chrome 117.0, Firefox 119.0]
Incoming: [Chrome 117.0, Firefox 119.0, Slack 4.0]
Expected delta:
- To add: 1 app (Slack)
- To remove: 0 apps
- Unchanged: 2 apps
Operations: 1 SELECT + 1 INSERT = 2 (vs 5 DELETE+INSERT before)
Reduction: 60%
Result: PASS
```
### Scenario 3: App Uninstalled
```
Current: [Chrome 117.0, Firefox 119.0, Office 16.0]
Incoming: [Chrome 117.0, Firefox 119.0]
Expected delta:
- To add: 0 apps
- To remove: 1 app (Office)
- Unchanged: 2 apps
Operations: 1 SELECT + 1 DELETE = 2 (vs 5 DELETE+INSERT before)
Reduction: 60%
Result: PASS
```
### Scenario 4: Version Update
```
Current: [Chrome 117.0, Firefox 119.0]
Incoming: [Chrome 117.0, Firefox 120.0]
Expected delta:
- To add: 1 app (Firefox 120.0)
- To remove: 1 app (Firefox 119.0)
- Unchanged: 1 app (Chrome)
Operations: 1 SELECT + 1 DELETE + 1 INSERT = 3 (vs 4 DELETE+INSERT before)
Reduction: 25%
Result: PASS
```
### Scenario 5: Mixed Changes
```
Current: [Chrome 117.0, Firefox 119.0, Office 16.0]
Incoming: [Chrome 117.0, Firefox 120.0, Slack 4.0]
Expected delta:
- To add: 2 apps (Firefox 120.0, Slack 4.0)
- To remove: 2 apps (Firefox 119.0, Office 16.0)
- Unchanged: 1 app (Chrome)
Operations: 1 SELECT + 2 DELETE + 2 INSERT = 5 (vs 6 DELETE+INSERT before)
Reduction: 17%
Result: PASS
```
---
## 3. Python Syntax Validation
### Files Checked
```
OK plugins/endpoints/plugin.py
OK plugins/endpoints/models/endpoint.py
OK plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py
OK scripts/backfill_app_fingerprints.py
Result: PASS (all files have valid Python syntax)
```
---
## 4. Safety Features Verification
### Feature 1: Row-Level Locking
```
Location: plugins/endpoints/plugin.py:1143
Code: .with_for_update()
Purpose: Prevents concurrent update races (TOCTOU)
Status: VERIFIED
```
### Feature 2: Fingerprint Format Validation
```
Location: plugins/endpoints/plugin.py:1008, 1122
Code: _is_valid_fingerprint(fp)
Purpose: Rejects malformed fingerprints (must be 64 hex chars)
Status: VERIFIED
```
### Feature 3: Delete Count Verification
```
Location: plugins/endpoints/plugin.py:1202-1209
Code: if deleted_count != len(to_remove): raise RuntimeError(...)
Purpose: Detects silent failures or orphaned records
Status: VERIFIED
```
### Feature 4: Transaction Flush (Early Error Detection)
```
Location: plugins/endpoints/plugin.py:1087, 1168, 1224
Code: db.session.flush()
Purpose: Catches constraint violations before commit
Status: VERIFIED
```
### Feature 5: Memory Limits
```
Location: plugins/endpoints/plugin.py:1042, 1146
Code: MAX_APPS_PER_ENDPOINT = 5000
Purpose: Prevents memory exhaustion on pathological cases
Status: VERIFIED
```
### Feature 6: Algorithm Verification (in backfill script)
```
Location: scripts/backfill_app_fingerprints.py
Code: verify_fingerprint_algorithm()
Purpose: Ensures backend matches agent fingerprint computation
Status: VERIFIED
```
### Feature 7: Metrics Logging
```
Location: plugins/endpoints/plugin.py:1049, 1090, 1170, 1227
Code: g.collector_metrics['applications'] = {...}
Purpose: Tracks adds/removes/unchanged for monitoring
Status: VERIFIED
```
### Feature 8: Legacy Fallback
```
Location: plugins/endpoints/plugin.py:1073-1103
Code: if not has_fingerprints: ... (delete-all-insert-all)
Purpose: Backward compatible with old agents
Status: VERIFIED
```
---
## 5. ADR Compliance Verification
### ADR-008: Plugin Migration Ownership
```
File: plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py
Location: Correct (plugins/endpoints/migrations/versions/)
Revision chain: 0020 revises 0019 (linear chain maintained)
Status: COMPLIANT
```
### Database Naming Conventions
```
Column: app_fingerprint (snake_case) - COMPLIANT
Index: idx_endpointapplications_fingerprint - COMPLIANT
Table: endpointapplications (existing, lowercase concatenated) - COMPLIANT
Status: COMPLIANT
```
### ADR-006: Collector Contract
```
Backward compatible: YES (legacy fallback for agents without fingerprints)
Forward compatible: YES (fingerprint field nullable)
Breaking changes: NONE
Status: COMPLIANT
```
### Code Style (CLAUDE.md)
```
Plain ASCII only: VERIFIED (no unicode/emojis)
Minimal comments: VERIFIED (code is self-documenting)
No LLM attribution: VERIFIED
Status: COMPLIANT
```
---
## 6. Performance Impact Analysis
### Baseline (Before Optimization)
```
Scenario: 200 endpoints, 50 apps each, reporting every 15 minutes
Operations per report: 50 DELETE + 50 INSERT = 100 operations
Reports per hour: 200 endpoints × 4 reports = 800 reports
Operations per hour: 800 × 100 = 80,000 operations
Operations per day: 80,000 × 24 = 1,920,000 operations
```
### After Optimization (Typical - 99% no changes)
```
Scenario: Same 200 endpoints, 50 apps each
Operations per report (no changes): 1 SELECT = 1 operation
Operations per report (1 change): 1 SELECT + 2 ops = 3 operations
Average operations per hour: ~800 operations (99% no-change)
Average operations per day: ~19,200 operations
Reduction: 1,920,000 → 19,200 = 99.0% reduction
```
### Worst Case (All Apps Changed Every Report)
```
Operations per report: 1 SELECT + 50 DELETE + 50 INSERT = 101 operations
Operations per day: 800 × 101 = 80,800 operations
Reduction vs baseline: 1,920,000 → 80,800 = 95.8% reduction
Note: This scenario is unrealistic (all 50 apps changing every 15 min)
Even in worst case, still 95%+ reduction due to SELECT optimization
```
---
## 7. Code Coverage Analysis
### Modified Files
```
1. plugins/endpoints/models/endpoint.py
- Lines added: 8 (app_fingerprint column, index)
- Lines modified: 0
- Status: COMPLETE
2. plugins/endpoints/plugin.py
- Lines added: 143
- Lines modified: 38
- Method: _replace_applications() completely rewritten
- Status: COMPLETE
3. plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py
- New file: 62 lines
- Status: COMPLETE
4. scripts/backfill_app_fingerprints.py
- New file: 168 lines
- Status: COMPLETE
5. docs/agent-app-fingerprints.md
- New file: 275 lines
- Status: COMPLETE
6. tests/test_plugins/test_endpoints_app_delta.py
- New file: 254 lines
- 15 test cases defined
- Status: COMPLETE (awaiting pytest installation to run)
```
---
## 8. Edge Cases Handled
### Edge Case 1: Empty Inventory
```
Scenario: Agent reports 0 apps (all uninstalled)
Handling: Deletes all existing records
Status: HANDLED (lines 1044-1056)
```
### Edge Case 2: Missing Displayname
```
Scenario: App entry without displayname
Handling: Skipped with warning
Status: HANDLED (lines 1119-1121)
```
### Edge Case 3: Invalid Fingerprint
```
Scenario: Malformed fingerprint (wrong length, non-hex)
Handling: Skipped with warning
Status: HANDLED (lines 1122-1128)
```
### Edge Case 4: String Truncation
```
Scenario: App name/version/publisher exceeds column length
Handling: Truncated to column limits (255/128/255)
Status: HANDLED (lines 1130-1134)
```
### Edge Case 5: Large Inventory (>5000 apps)
```
Scenario: Endpoint with >5000 apps
Handling: Falls back to legacy mode (delete-all-insert-all)
Status: HANDLED (lines 1146-1183)
```
### Edge Case 6: Legacy Agent (No Fingerprints)
```
Scenario: Old agent doesn't send fingerprints
Handling: Falls back to delete-all-insert-all
Status: HANDLED (lines 1073-1103)
```
### Edge Case 7: Concurrent Updates
```
Scenario: Two agents report simultaneously for same asset
Handling: Row-level locking (.with_for_update())
Status: HANDLED (line 1143)
```
### Edge Case 8: Delete Failure
```
Scenario: Expected to delete 10 records, only deleted 8
Handling: Raises RuntimeError with details
Status: HANDLED (lines 1202-1209)
```
---
## 9. Known Limitations
1. **SHA256 Collision**: Astronomically unlikely (2^-256) but theoretically possible
- Impact: Two different apps could have same fingerprint
- Mitigation: Probability is negligible in practice
2. **Memory Limit**: Endpoints with >5000 apps fall back to legacy mode
- Impact: Large inventories don't benefit from optimization
- Mitigation: Fallback ensures functionality, very rare in practice
3. **Backward Compatibility Window**: During rollout, mixed agent versions
- Impact: Some reports use legacy mode, some use delta mode
- Mitigation: Temporary, resolves after full agent deployment
---
## 10. Recommendations
### Before Production Deployment
1. Install pytest and run full test suite: `pytest tests/test_plugins/test_endpoints_app_delta.py -v`
2. Run migration on staging environment first
3. Run backfill script with --dry-run to verify fingerprint computation
4. Monitor logs for any unexpected warnings during rollout
### During Rollout
1. Deploy to 10% of fleet first, monitor for 24 hours
2. Check logs for "delta mode" vs "legacy mode" distribution
3. Verify database write volume reduction in monitoring
4. Gradually roll out to remaining fleet
### Post-Deployment
1. Monitor for any "Delete count mismatch" errors (indicates race condition)
2. Track fingerprint adoption rate (% of records with fingerprints)
3. Measure actual performance improvement (queries/sec reduction)
4. After 100% adoption, consider making app_fingerprint NOT NULL
---
## Conclusion
**Overall Status**: READY FOR DEPLOYMENT
All tests passed, safety features verified, ADR compliance confirmed.
**Expected Impact**:
- 99% reduction in database writes (1.92M/day → 19K/day)
- Faster check-in processing (100 ops → 1-3 ops)
- Full backward compatibility maintained
- Comprehensive safety features protect against edge cases
**Next Steps**:
1. Install pytest: `pip install pytest pytest-flask`
2. Run full test suite
3. Deploy to staging
4. Monitor and adjust as needed
5. Deploy to production with phased rollout

258
VALIDATION-FIXES-2026-09-14.md
View File

@@ -1,258 +0,0 @@
# Validation Fixes Applied - 2026-09-14
**Status**: All CRITICAL and MEDIUM issues from validation review FIXED
---
## Summary
Fable validation identified 1 CRITICAL blocker and 2 MEDIUM issues in the initial critical fixes. All have been resolved.
---
## Fix #1: Index Definition Mismatch (CRITICAL)
**File**: `plugins/endpoints/models/endpoint.py`
**Lines**: 471, 475
**Problem**: Migration created UNIQUE index, but model declared non-unique index. This would cause schema drift and IntegrityError not being handled correctly.
**Fix Applied**:
```python
# Line 471 - Removed redundant index=True (creates single-column index)
app_fingerprint = db.Column(db.String(64), nullable=True)
# Line 475 - Added unique=True to match migration
db.Index('idx_endpointapplications_fingerprint', 'assetid', 'app_fingerprint', unique=True),
```
**Impact**:
- Model now matches migration schema
- UNIQUE constraint enforced at model level
- No schema drift between database and SQLAlchemy
- IntegrityError will be raised if duplicate fingerprints attempted
---
## Fix #2: Redundant Single-Column Index (MEDIUM)
**File**: `plugins/endpoints/models/endpoint.py`
**Line**: 471
**Problem**: `index=True` on column definition created single-column index on `app_fingerprint`, separate from composite index `(assetid, app_fingerprint)`. This wasted disk space and write performance.
**Fix Applied**: Removed `index=True` from column definition (see Fix #1 above)
**Impact**:
- Only one index now: composite (assetid, app_fingerprint)
- Reduced disk space usage
- Faster INSERT/UPDATE/DELETE (only one index to maintain)
- Query performance unchanged (composite index covers single-column queries on assetid)
**Migration Note**: Existing single-column index will need to be dropped manually or via migration:
```python
# Future migration to clean up
op.drop_index('ix_endpointapplications_app_fingerprint', 'endpointapplications')
```
---
## Fix #3: Advisory Lock for Application Delta (MEDIUM)
**File**: `plugins/endpoints/plugin.py`
**Lines**: 1140-1151
**Problem**: Application delta used `with_for_update()` but not advisory lock like event deduplication does. Row-level locks only lock EXISTING rows, allowing race condition if another transaction commits changes between query and modifications.
**Fix Applied**:
```python
# Added advisory lock BEFORE loading current_apps
from sqlalchemy import text
db.session.execute(
text('SELECT pg_advisory_xact_lock(:lock_key)'),
{'lock_key': assetid}
)
# Then proceed with SELECT FOR UPDATE
current_apps = (EndpointApplication.query
.filter_by(assetid=assetid)
.limit(MAX_APPS_PER_ENDPOINT + 1)
.with_for_update()
.all())
```
**Impact**:
- Eliminates race condition between concurrent reports for same asset
- Advisory lock held until transaction commit
- Only one agent can update applications for an asset at a time
- Delete count verification (line 1206) will no longer trigger false positives from races
- UNIQUE constraint (Fix #1) + advisory lock = complete race protection
**Performance**: Minimal overhead (in-memory lock lookup, ~1 microsecond)
---
## Fix #4: Unicode Characters in Migration (CODE STYLE)
**File**: `plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py`
**Line**: 11
**Problem**: Used multiplication sign (×) and arrow (->) instead of ASCII equivalents
**Fix Applied**:
```python
# Before: 200 endpoints × 50 apps × 4 reports/hour
# After: 200 endpoints x 50 apps x 4 reports/hour
# Before: 1.92M writes/day → 1,920 writes/day
# After: 1.92M writes/day -> 1,920 writes/day
```
**Impact**: Complies with CLAUDE.md requirement for plain ASCII only
---
## Fix #5: Unicode Arrow in Backfill Script (CODE STYLE)
**File**: `scripts/backfill_app_fingerprints.py`
**Line**: 154
**Problem**: Used right arrow (→) instead of ASCII equivalent
**Fix Applied**:
```python
# Before: print(f" {app.displayname:50s} → {fp[:16]}...")
# After: print(f" {app.displayname:50s} -> {fp[:16]}...")
```
**Impact**: Complies with CLAUDE.md requirement for plain ASCII only
---
## ADR Compliance Verification
All ADRs now fully compliant:
**ADR-008: Plugin Migration Ownership** ✓
- Migration in correct location
- Linear chain maintained (0020 revises 0019)
- plugin_name set correctly
**Database Naming Conventions** ✓
- Table: `endpointapplications` (lowercase concatenated)
- Column: `app_fingerprint` (snake_case)
- Index: `idx_endpointapplications_fingerprint` (follows convention)
- UNIQUE constraint properly declared in both migration and model
**ADR-006: Collector Contract** ✓
- Backward compatible (nullable column, legacy fallback)
- No breaking changes
**Code Style (CLAUDE.md)** ✓
- Plain ASCII only (all unicode removed)
- No emojis or em-dashes
- Minimal comments
- No LLM attribution in code
---
## Files Modified
1. **plugins/endpoints/models/endpoint.py** (2 changes)
- Removed `index=True` from app_fingerprint column (line 471)
- Added `unique=True` to composite index (line 475)
2. **plugins/endpoints/plugin.py** (1 change)
- Added advisory lock before loading current_apps (lines 1140-1151)
3. **plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py** (1 change)
- Replaced unicode symbols with ASCII equivalents (line 11, 13)
4. **scripts/backfill_app_fingerprints.py** (1 change)
- Replaced unicode arrow with ASCII (line 154)
---
## Validation Summary
**Initial Review (Round 1)**:
- 4 critical issues found
- All 4 fixed
**Validation Review (Round 2)**:
- 1 CRITICAL issue found (index mismatch)
- 2 MEDIUM issues found (redundant index, no advisory lock for apps)
- 3 LOW issues found (test vectors, strict delete check, fallback edge case)
- All CRITICAL and MEDIUM issues fixed
**Final Status**: READY FOR TESTING
---
## Outstanding LOW Priority Issues
These are acceptable for initial deployment but should be addressed in future:
**L1. Test Vectors Have No Expected Values**
- `expected_fp: None` in backfill script
- Recommendation: Populate with actual SHA256 hashes for regression detection
**L2. Delete Count Verification Too Strict**
- RuntimeError on any mismatch may cause false positives
- Recommendation: Log warning instead of raising for small mismatches (<5)
**L3. Memory Limit Fallback Edge Case**
- Pathological case (>5000 apps) uses legacy mode without advisory lock
- Impact: Minimal (ShopDB endpoints typically have 50-200 apps)
- Recommendation: Add advisory lock to fallback path for completeness
---
## Testing Checklist Before Production
- [ ] Verify Python syntax: `python -m py_compile plugins/endpoints/plugin.py plugins/endpoints/models/endpoint.py`
- [ ] Run test suite: `pytest tests/test_plugins/test_endpoints_app_delta.py -v`
- [ ] Run style check: `scripts/check-naming-and-style.sh`
- [ ] Test migration upgrade: `flask db upgrade` on staging
- [ ] Test migration downgrade: `flask db downgrade -1` on staging
- [ ] Test backfill script: `python scripts/backfill_app_fingerprints.py --dry-run`
- [ ] Test concurrent updates: 2 agents reporting same asset simultaneously
- [ ] Verify no IntegrityError in logs
- [ ] Verify delete count verification works correctly
- [ ] Check for unicode violations: `grep -P "[^\x00-\x7F]" plugins/endpoints/plugin.py` (should be empty)
---
## Deployment Sequence
1. Deploy migration: `flask db upgrade`
2. Drop old single-column index (if exists): `DROP INDEX IF EXISTS ix_endpointapplications_app_fingerprint;`
3. Run backfill: `python scripts/backfill_app_fingerprints.py`
4. Deploy updated plugin.py
5. Deploy updated C# agent with fingerprint computation
6. Monitor logs for delta mode metrics
---
## Performance Impact (Updated)
**Before**: 1.92M writes/day
**After**: 1,920 writes/day
**Reduction**: 99.9%
**Advisory Lock Overhead**: <1 microsecond per transaction (negligible)
---
## Conclusion
All critical and medium issues from validation review have been fixed. The code now:
- ✓ Matches migration schema (UNIQUE constraint)
- ✓ Has no redundant indexes
- ✓ Uses advisory locks for both events and applications
- ✓ Complies with all ADRs
- ✓ Uses plain ASCII only
- ✓ Has comprehensive race condition protection
**Status**: READY FOR TESTING AND STAGING DEPLOYMENT

225
VERSION-UPDATE-2026-09-14.md
View File

@@ -1,225 +0,0 @@
# Version Update - 2026-09-14
**Status**: Versions incremented and deployed successfully ✓
---
## Version Changes
### ShopDB Endpoint Agent
- **Previous**: 1.16.0
- **New**: 1.17.0
- **Type**: Minor version bump
- **Reason**: Critical race condition fixes and application delta optimization backend
### ShopDB Endpoint Tray
- **Previous**: 1.6.8
- **New**: 1.6.9
- **Type**: Patch version bump
- **Reason**: Coordinated release with agent 1.17.0 (no tray code changes)
---
## Changelog for Agent 1.17.0
### Critical Fixes
1. **Advisory Locks for Race Prevention**
- Application delta updates now use `pg_advisory_xact_lock(assetid)`
- Event deduplication now uses `pg_advisory_xact_lock(assetid)`
- Prevents TOCTOU race conditions in both code paths
- Lock held until transaction commit
2. **UNIQUE Constraint on Fingerprints**
- Index `idx_endpointapplications_fingerprint` now has `unique=True`
- Prevents duplicate (assetid, app_fingerprint) pairs
- Ensures delta logic assumption of one fingerprint per app per asset
3. **Pre-compiled Regex Validation**
- Fingerprint validation regex compiled once at module load
- 5-10% faster validation (40,000+ validations/hour)
- No functional change, performance optimization
4. **Code Style Compliance**
- All unicode characters replaced with ASCII equivalents
- Complies with CLAUDE.md project standards
- Model/migration schema alignment
### Backend Features (Agent Side Not Yet Implemented)
- **Application Fingerprint Column**: `app_fingerprint` (SHA256 hash)
- **Delta-Based Updates**: Only changed apps are touched (99% write reduction)
- **Backward Compatibility**: Legacy agents without fingerprints use delete-all-insert-all fallback
- **Migration**: 0020_app_delta_fingerprints.py adds column and UNIQUE index
- **Backfill Script**: Computes fingerprints for existing records
**Note**: This release includes only the backend infrastructure. Agent-side fingerprint computation will be implemented in a future release. Current behavior: agents report without fingerprints, backend uses legacy fallback mode.
---
## Deployment Verification
### VM: 192.168.100.10 (SFTest)
**Service Status**:
```
SERVICE_NAME: ShopDbEndpointAgent
STATE : 4 RUNNING
WIN32_EXIT_CODE : 0 (0x0)
```
**Agent Logs** (Latest):
```
2026-09-14 05:15:24.046 [INF] TelemetryWorker: Endpoint agent 1.17.0.0 -> https://192.168.100.1:5001, cadence 900s +/-60s
2026-09-14 05:15:24.058 [INF] CommandWorker: Command channel started for host DESKTOP-HF1K9M4
2026-09-14 05:15:24.063 [INF] Lifetime: Application started. Hosting environment: Production
```
**Version Confirmed**: 1.17.0.0 ✓
### File Sizes
- `shopdbagent.exe`: 35 MB (36,118,519 bytes)
- `shopdb-endpoint-tray.exe`: 69 MB (72,030,295 bytes)
---
## Version History Context
### Agent Version Timeline
- **1.16.0**: Last stable release (fleet alert scope fixes)
- **1.17.0**: Application delta optimization backend + critical race fixes (this release)
### Tray Version Timeline
- **1.6.8**: Coordinated with agent 1.14.2 (no tray code changes)
- **1.6.9**: Coordinated with agent 1.17.0 (no tray code changes)
---
## Project Files Modified
### Version Bumps
1. `agent/ShopDbEndpointAgent/ShopDbEndpointAgent.csproj` (line 236)
- Changed: `<Version>1.16.0</Version>`
- To: `<Version>1.17.0</Version>`
- Added changelog comment explaining changes
2. `agent/ShopDbEndpointAgent.Tray/ShopDbEndpointAgent.Tray.csproj` (line 95)
- Changed: `<Version>1.6.8</Version>`
- To: `<Version>1.6.9</Version>`
- Added changelog comment for coordinated release
### Backend Changes (Already Applied)
- `plugins/endpoints/plugin.py` (+291 lines)
- `plugins/endpoints/models/endpoint.py` (+7 lines)
- `plugins/endpoints/migrations/versions/0020_app_delta_fingerprints.py` (new)
- `scripts/backfill_app_fingerprints.py` (new)
---
## Build Information
**Date**: 2026-09-14 08:14 AM
**Build Configuration**: Release
**Platform**: win-x64
**Self-Contained**: Yes
**.NET SDK**: 8.0.424
**Build Times**:
- ShopDbEndpointAgent v1.17.0: ~15 seconds
- ShopDbEndpointAgent.Tray v1.6.9: ~7 seconds
**Build Status**: Succeeded, 0 warnings, 0 errors ✓
---
## Deployment Sequence
1. ✓ Incremented versions in .csproj files
2. ✓ Rebuilt both projects (Release configuration)
3. ✓ Published to agent/publish/ directory
4. ✓ Stopped ShopDbEndpointAgent service on VM
5. ✓ Transferred new executables via HTTP (192.168.100.1:8080)
6. ✓ Replaced executables in C:\ProgramData\ShopDB\EndpointAgent\
7. ✓ Started ShopDbEndpointAgent service
8. ✓ Verified version 1.17.0.0 in logs
---
## Testing Checklist
### Immediate Testing
- [x] Service starts successfully
- [x] Version 1.17.0.0 appears in logs
- [x] Agent reports telemetry to server
- [ ] Monitor for legacy fallback mode (expected: all reports use legacy until C# fingerprints implemented)
- [ ] Check for errors in logs over next 24 hours
### Backend Testing (Requires Migration Deployment)
- [ ] Deploy migration 0020 to staging
- [ ] Run backfill script
- [ ] Verify app_fingerprint column populated for existing records
- [ ] Monitor for IntegrityError (should be 0 with UNIQUE constraint)
- [ ] Monitor for RuntimeError from delete count mismatch (should be 0 with advisory locks)
### Future Testing (After C# Fingerprint Implementation)
- [ ] Agent computes SHA256 fingerprints
- [ ] Backend detects fingerprints and uses delta mode
- [ ] Verify 99% write reduction
- [ ] Monitor fingerprint adoption rate (should reach 100%)
---
## Next Release Plan
### Agent 1.18.0 (Future)
- Implement C# fingerprint computation
- SHA256(displayname|displayversion|publisher)
- Send fingerprints in collector payload
- Enable delta mode on backend
- Expected impact: 99% database write reduction
### Migration Path
1. Deploy agent 1.17.0 (backend ready, agent uses legacy)
2. Deploy migration 0020 (add column, run backfill)
3. Deploy agent 1.18.0 (agent sends fingerprints)
4. Monitor adoption (% of reports with fingerprints)
5. After 100% adoption, consider making column NOT NULL
---
## Rollback Procedure
If issues arise with version 1.17.0:
1. **Stop service on VM**:
```
sc.exe stop ShopDbEndpointAgent
```
2. **Restore previous version executables** (v1.16.0):
```
copy /Y C:\Backups\shopdbagent-1.16.0.exe C:\ProgramData\ShopDB\EndpointAgent\shopdbagent.exe
copy /Y C:\Backups\shopdb-endpoint-tray-1.6.8.exe C:\ProgramData\ShopDB\EndpointAgent\shopdb-endpoint-tray.exe
```
3. **Start service**:
```
sc.exe start ShopDbEndpointAgent
```
4. **Verify rollback in logs**:
```
powershell -Command "Get-Content C:\ProgramData\ShopDB\EndpointAgent\logs\*.log | Select-Object -Last 5"
```
**Note**: Backend changes are backward compatible. Rolling back agent does not require backend rollback.
---
## Summary
Successfully incremented versions to reflect critical backend fixes:
- Agent: 1.16.0 → 1.17.0 (minor bump for race fixes)
- Tray: 1.6.8 → 1.6.9 (patch bump, coordinated release)
Deployed to SFTest VM and verified version 1.17.0.0 running successfully. Backend infrastructure ready for future agent-side fingerprint implementation.
**Status**: DEPLOYMENT COMPLETE WITH CORRECT VERSIONS ✓

199
agent/INSTALL-ON-VM.md
View File

@@ -1,199 +0,0 @@
# Install ShopDB Agent on Hyper-V VM
This guide shows how to install the fleet agent on your Hyper-V VM to report back to ShopDB running on the host PC.
## Configuration for Your Setup
**ShopDB Server (Host PC):**
- URL: `https://172.31.96.1:5001`
- Collector API Key: `shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43`
**VM Network:**
- VM can reach host at: `172.31.96.1`
- Host's Hyper-V virtual adapter IP
---
## Step 1: Build the Agent (On Host PC)
**Install .NET SDK first:**
```powershell
# On Host PC (this machine):
winget install Microsoft.DotNet.SDK.8
```
**Build agent executables:**
```powershell
cd C:\Users\270015376\Documents\github\flask-shopdb\agent
# Build service
dotnet publish ShopDbEndpointAgent -c Release -r win-x64 --self-contained -o C:\temp\shopdb-agent
# Build tray app (optional)
dotnet publish ShopDbEndpointAgent.Tray -c Release -r win-x64 --self-contained -o C:\temp\shopdb-agent-tray
# Package for VM
Compress-Archive -Path C:\temp\shopdb-agent -DestinationPath C:\temp\shopdb-agent.zip
```
---
## Step 2: Copy to VM
**Copy the ZIP to your VM:**
- Via RDP shared folder
- Via network share
- Via USB
**Extract on VM:**
```powershell
# On VM:
Expand-Archive C:\path\to\shopdb-agent.zip -DestinationPath C:\ProgramData\ShopDB\EndpointAgent
```
---
## Step 3: Configure Agent (On VM)
**Set registry configuration:**
```powershell
# Run as Administrator on VM:
New-Item -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Force | Out-Null
# Point to HOST PC's IP (not localhost!)
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'BaseUrl' -Value 'https://172.31.96.1:5001'
# Use the collector key
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'CollectorKey' -Value 'shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43'
```
---
## Step 4: Test Agent (On VM)
**Test one report:**
```powershell
cd C:\ProgramData\ShopDB\EndpointAgent
.\shopdb-endpoint-agent.exe --once
```
**Expected output:**
```
[INFO] Agent starting...
[INFO] Collecting endpoint state...
[INFO] Reporting to https://172.31.96.1:5001/api/collector/endpoints
[INFO] Report successful
```
**If you see SSL certificate errors:**
The self-signed cert will cause warnings. You can either:
- Ignore them (agent still works)
- Or use HTTP (not recommended): `http://172.31.96.1:5001`
---
## Step 5: Install as Service (On VM)
**Create Windows service:**
```powershell
# Run as Administrator on VM:
sc.exe create "ShopDBEndpointAgent" `
binPath= "C:\ProgramData\ShopDB\EndpointAgent\shopdb-endpoint-agent.exe" `
start= auto `
DisplayName= "ShopDB Endpoint Agent"
sc.exe description "ShopDBEndpointAgent" "Reports endpoint telemetry to ShopDB server."
# Start the service
sc.exe start "ShopDBEndpointAgent"
```
**Check service status:**
```powershell
Get-Service ShopDBEndpointAgent
```
---
## Step 6: Verify on Host PC
**Check ShopDB received the report:**
1. Open: `https://localhost:5001/computers`
2. Look for your VM's hostname
3. Click on it to see endpoint data
**Or check logs:**
```bash
# On host PC:
docker logs flask-shopdb-api-1 | grep -i "collector/endpoints"
```
---
## Troubleshooting
### Agent can't reach server
**Test connectivity from VM:**
```powershell
# On VM:
Test-NetConnection -ComputerName 172.31.96.1 -Port 5001
```
**If blocked by firewall:**
```powershell
# On Host PC, allow inbound:
New-NetFirewallRule -DisplayName "ShopDB API" -Direction Inbound -LocalPort 5001 -Protocol TCP -Action Allow
```
### SSL certificate errors
The agent works even with self-signed cert warnings. To eliminate them:
**Option 1: Use HTTP (dev only)**
```powershell
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'BaseUrl' -Value 'http://172.31.96.1:5001'
```
**Option 2: Trust the cert**
- Export cert from host
- Import to VM's Trusted Root
### Agent reports but PC doesn't appear
**Check collector key matches:**
```powershell
# On VM:
Get-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'CollectorKey'
# Should match what's in ShopDB .env:
# shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43
```
---
## Uninstall
```powershell
# Stop and remove service
sc.exe stop "ShopDBEndpointAgent"
sc.exe delete "ShopDBEndpointAgent"
# Remove files
Remove-Item -Recurse C:\ProgramData\ShopDB\EndpointAgent
# Remove registry
Remove-Item -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Recurse
```
---
## Quick Reference
**Host IP:** `172.31.96.1`
**ShopDB URL:** `https://172.31.96.1:5001`
**Collector Key:** `shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43`
**Agent Location:** `C:\ProgramData\ShopDB\EndpointAgent\`
**Service Name:** `ShopDBEndpointAgent`
**Report Interval:** 15 minutes (configurable in appsettings.json)

51
agent/QUICK-FIX-VM.md
View File

@@ -1,51 +0,0 @@
# Quick Fix: Agent Can't Connect from VM
## Problem
SSL certificate errors when connecting from VM to host.
## Solution: Use HTTP on Local Network
Since the VM and host are on a trusted local network (Hyper-V), use HTTP instead of HTTPS.
### On VM - Set HTTP in Registry:
```powershell
# On VM (as Administrator):
New-Item -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Force
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'BaseUrl' -Value 'http://172.31.96.1:5001'
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'CollectorKey' -Value 'shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43'
```
### Test it works:
```powershell
# On VM:
curl.exe http://172.31.96.1:5001
# Should return HTML
```
### Then run the agent:
```powershell
cd C:\ProgramData\ShopDB\EndpointAgent
.\shopdb-endpoint-agent.exe --once
```
## Why HTTP is OK Here
- [X] VM and host on same physical machine (Hyper-V)
- [X] Traffic never leaves the host
- [X] Hyper-V internal network is isolated
- [X] No external network exposure
For production multi-machine deployment, use HTTPS with proper certificates.
## Alternative: Fix HTTPS (More Complex)
If you need HTTPS:
1. Export the self-signed cert from host
2. Import to VM's Trusted Root Certification Authorities
3. Use HTTPS URL
See main INSTALL-ON-VM.md for details.

281
plugins/computers/client/Report-AssetToShopDB.ps1
View File

@@ -17,11 +17,18 @@
# to Asset.assetnumber; the placeholder 9999 is skipped server-side.
#
# AUTH: the collector API does NOT honor the GE-Enforce IP allowlist (that only
# covers the geenforce fetch/report endpoints). It needs a collector.ingest key,
# sent as the X-API-Key header. The key is read from HKLM:\SOFTWARE\GE\ShopDB
# CollectorKey (same secret store the GE-Enforce client uses; provisioned at
# imaging), or overridden via the manifest entry's Args -ApiKey. Never bake the
# key into the manifest JSON on the share.
# covers the geenforce fetch/report endpoints). Once the endpoint agent (1.20+)
# has enrolled this PC, its per-PC credential is read from the agent's
# SYSTEM-only credential.bin and sent as X-Endpoint-Credential; the server then
# refuses the shared key for this PC (401 SHARED_KEY_REFUSED). Before that, a
# collector.ingest key is sent as the X-API-Key header, read from
# HKLM:\SOFTWARE\GE\ShopDB CollectorKey (same secret store the GE-Enforce client
# uses; provisioned at imaging), or overridden via the manifest entry's Args
# -ApiKey. Never bake the key into the manifest JSON on the share.
#
# PINS: when HKLM:\SOFTWARE\GE\ShopDB ServerPins is set, the server's TLS key
# must match one of them or nothing is sent. See
# plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md.
#
# Deployed in common\ (runs on EVERY shopfloor pc-type). Non-DNC bays report
# identity with no machineNo, so no PC-to-machine link is built - by design.
@@ -70,6 +77,227 @@ function Log([string]$msg) {
"$ts $msg" | Tee-Object -FilePath $logFile -Append | Out-Null
}
# ---------------------------------------------------------------------------
# Per-PC credential and server pin. Kept in step with the same block in
# Report-AssetToShopDB.ps1, Report-PrintersToShopDB.ps1 and
# upgrade-shopdbagent.ps1: each runs standalone, so the text is duplicated.
#
# Credential: the endpoint agent (1.20+) enrolls and stores its own credential
# in credential.bin, DPAPI LocalMachine, readable only by SYSTEM and
# Administrators. When present it is sent as X-Endpoint-Credential and the
# shared key is not sent. Without it the shared CollectorKey goes as X-API-Key.
#
# Pins: HKLM\SOFTWARE\GE\ShopDB ServerPins (REG_MULTI_SZ), lowercase hex
# SHA-256 of the server leaf certificate's SubjectPublicKeyInfo. When any pin
# is set the TLS handshake is accepted only if the leaf key matches one.
# ---------------------------------------------------------------------------
$ShopDbCredentialFile = Join-Path $env:ProgramData 'ShopDB\EndpointAgent\credential\credential.bin'
$ShopDbPinSource = @'
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
public static class ShopDbServerPinValidator
{
private static readonly HashSet<string> Pins = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
public static string LastPresented;
public static void Install(string[] pins)
{
Pins.Clear();
foreach (string pin in pins)
{
if (!String.IsNullOrEmpty(pin)) { Pins.Add(pin); }
}
ServicePointManager.ServerCertificateValidationCallback = Validate;
}
public static bool Validate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors errors)
{
if (Pins.Count == 0) { return errors == SslPolicyErrors.None; }
if (certificate == null) { return false; }
try
{
LastPresented = SpkiSha256Hex(certificate.GetRawCertData());
}
catch (Exception)
{
LastPresented = null;
return false;
}
return Pins.Contains(LastPresented);
}
// Walks the DER: Certificate > TBSCertificate > [0] version (optional),
// serialNumber, signature, issuer, validity, subject, subjectPublicKeyInfo.
public static string SpkiSha256Hex(byte[] der)
{
int length;
int position = EnterSequence(der, 0, out length);
position = EnterSequence(der, position, out length);
if (der[position] == 0xA0) { position = Skip(der, position); }
for (int field = 0; field < 5; field++) { position = Skip(der, position); }
if (der[position] != 0x30) { throw new FormatException("SubjectPublicKeyInfo not found"); }
int end = Skip(der, position);
using (SHA256 sha = SHA256.Create())
{
byte[] hash = sha.ComputeHash(der, position, end - position);
return BitConverter.ToString(hash).Replace("-", "").ToLowerInvariant();
}
}
private static int ReadLength(byte[] der, int position, out int length)
{
int first = der[position];
if (first < 0x80) { length = first; return position + 1; }
int count = first & 0x7F;
if (count == 0 || count > 4) { throw new FormatException("Unsupported DER length"); }
length = 0;
for (int i = 1; i <= count; i++) { length = (length << 8) | der[position + i]; }
return position + 1 + count;
}
private static int EnterSequence(byte[] der, int position, out int length)
{
if (der[position] != 0x30) { throw new FormatException("Expected SEQUENCE"); }
return ReadLength(der, position + 1, out length);
}
private static int Skip(byte[] der, int position)
{
int length;
int content = ReadLength(der, position + 1, out length);
return content + length;
}
}
'@
function Get-ShopDbServerPins {
# Both registry views, like CollectorKey. Anything not 64 hex chars is dropped.
$pins = @()
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64, [Microsoft.Win32.RegistryView]::Registry32)) {
try {
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {
$key = $base.OpenSubKey('SOFTWARE\GE\ShopDB')
if ($key) {
try {
foreach ($value in @($key.GetValue('ServerPins'))) {
if ($null -eq $value) { continue }
$pin = ([string]$value -replace '[\s:]', '').ToLowerInvariant()
if ($pin -match '^[0-9a-f]{64}$' -and $pins -notcontains $pin) { $pins += $pin }
}
} finally { $key.Dispose() }
}
} finally { $base.Dispose() }
} catch {}
}
return ,$pins
}
function Enable-ShopDbServerPins([string[]]$Pins) {
# True when pinning is active. False means pins are set but could not be
# enforced; the caller must not send anything then.
if ($PSVersionTable.PSEdition -eq 'Core') {
# PowerShell 7 web cmdlets ignore ServicePointManager, so the check
# would silently not happen.
Log 'ERROR ServerPins is set; pin checking needs Windows PowerShell 5.1 (powershell.exe), not pwsh'
return $false
}
try {
if (-not ('ShopDbServerPinValidator' -as [type])) {
Add-Type -TypeDefinition $ShopDbPinSource -ErrorAction Stop
}
[ShopDbServerPinValidator]::Install($Pins)
return $true
} catch {
Log "ERROR ServerPins is set but pin checking could not be enabled: $($_.Exception.Message)"
return $false
}
}
function Get-ShopDbEndpointCredential([string]$TargetUrl) {
# Empty when there is none or it cannot be used for this URL.
if (-not (Test-Path -LiteralPath $ShopDbCredentialFile)) { return '' }
try {
Add-Type -AssemblyName System.Security -ErrorAction Stop
$protected = [System.IO.File]::ReadAllBytes($ShopDbCredentialFile)
$plain = [System.Security.Cryptography.ProtectedData]::Unprotect(
$protected, $null, [System.Security.Cryptography.DataProtectionScope]::LocalMachine)
$record = [System.Text.Encoding]::UTF8.GetString($plain) | ConvertFrom-Json
} catch {
Log "WARN could not read the endpoint credential ($ShopDbCredentialFile): $($_.Exception.Message)"
return ''
}
$credential = [string]$record.credential
if (-not $credential.StartsWith('sdbe_')) {
Log 'WARN endpoint credential file has no usable credential; ignoring it'
return ''
}
if (-not $TargetUrl.StartsWith('https://', [System.StringComparison]::OrdinalIgnoreCase)) {
Log "WARN not sending the endpoint credential over a non-https URL ($TargetUrl)"
return ''
}
# Never hand a credential to a server other than the one that issued it.
$issuer = ([string]$record.baseurl).Trim().TrimEnd('/')
if ($issuer -and -not ($TargetUrl.TrimEnd('/') + '/').StartsWith($issuer + '/', [System.StringComparison]::OrdinalIgnoreCase)) {
Log "WARN endpoint credential was issued by $issuer, not the target $TargetUrl; not sending it"
return ''
}
return $credential
}
function Get-ShopDbHttpFailure($ErrorRecord) {
# Status and error envelope code (data.error.code) from a failed request.
$status = 0
$code = ''
$text = ''
try { $status = [int]$ErrorRecord.Exception.Response.StatusCode } catch {}
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
$text = $ErrorRecord.ErrorDetails.Message
} else {
try {
$reader = New-Object System.IO.StreamReader($ErrorRecord.Exception.Response.GetResponseStream())
$text = $reader.ReadToEnd()
$reader.Dispose()
} catch {}
}
if ($text) {
try { $code = [string](($text | ConvertFrom-Json).data.error.code) } catch {}
}
$presented = ''
if ('ShopDbServerPinValidator' -as [type]) { $presented = [string][ShopDbServerPinValidator]::LastPresented }
return @{ Status = $status; Code = $code; PresentedPin = $presented }
}
function Write-ShopDbAuthFailure($Failure) {
switch ($Failure.Code) {
'CREDENTIAL_INVALID' {
Log ('ERROR 401 CREDENTIAL_INVALID: the server does not accept this PC''s endpoint credential ' +
'(revoked or unknown). The endpoint agent deletes it and re-enrolls on its own.')
}
'SHARED_KEY_REFUSED' {
Log ('ERROR 401 SHARED_KEY_REFUSED: the server refuses the shared key for this PC, because ' +
'the PC is enrolled or the shared key is switched off. Reports need the endpoint agent ' +
"credential: check agent 1.20+ is installed, running and enrolled ($ShopDbCredentialFile).")
}
'ASSET_MISMATCH' {
Log ('ERROR 403 ASSET_MISMATCH: this PC''s credential belongs to a different asset than the ' +
'hostname in the report. Check the hostname and the enrollment in ShopDB.')
}
default {
if ($Failure.Status -eq 0 -and $Failure.PresentedPin) {
Log "ERROR TLS refused: server key pin $($Failure.PresentedPin) is not in ServerPins"
}
}
}
}
# Server from the GE-Enforce config hive when not passed via Args. Any site
# running this script is running the enforcement client, which cannot work
# without BaseUrl, so it is present wherever this is deployed.
@@ -102,11 +330,21 @@ if (-not $ApiKey) {
} catch {}
}
}
if (-not $ApiKey) {
Log 'ERROR no collector key (HKLM:\SOFTWARE\GE\ShopDB CollectorKey or -ApiKey). Skipping.'
$EndpointCredential = Get-ShopDbEndpointCredential $ApiUrl
if (-not $EndpointCredential -and -not $ApiKey) {
Log 'ERROR no endpoint credential and no collector key (HKLM:\SOFTWARE\GE\ShopDB CollectorKey or -ApiKey). Skipping.'
exit 0
}
$serverPins = Get-ShopDbServerPins
if ($serverPins.Count -gt 0) {
if (-not (Enable-ShopDbServerPins $serverPins)) {
Log 'ERROR refusing to report without the server pin check. Skipping.'
exit 0
}
Log "server pin check on ($($serverPins.Count) pin(s))"
}
# -AllowedRanges and the CollectorRanges registry value are ACCEPTED AND
# IGNORED. Deciding which NIC is the PC's address moved to the server, where
# the site's addressing actually lives: this script now reports every NIC and
@@ -326,14 +564,27 @@ $json = $body | ConvertTo-Json -Compress -Depth 4
Log ("POST {0} host={1} serial={2} pcType={3} make={4} model={5} os={6} boot={7} machineNo={8} user={9} ip={10} deviceId={11}" -f `
$ApiUrl, $hostname, $serialNumber, $pcType, $manufacturer, $model, $osVersion, $lastBootTime, $machineNo, $loggedInUser, $corpIp, $deviceId)
try {
$resp = Invoke-RestMethod -Uri $ApiUrl -Method Post -Body $json `
-ContentType 'application/json' `
-Headers @{ 'X-API-Key' = $ApiKey } `
-TimeoutSec $TimeoutSec -ErrorAction Stop
Log ("RESPONSE {0}" -f ($resp | ConvertTo-Json -Compress -Depth 4))
} catch {
Log "ERROR POST failed: $($_.Exception.Message)"
# Credential first; the shared key only when there is no credential, or when
# the server has just refused the credential (revoked: the shared key is
# admitted again for this PC until the agent re-enrolls). Never both at once.
$attempts = @()
if ($EndpointCredential) { $attempts += ,@('credential', @{ 'X-Endpoint-Credential' = $EndpointCredential }) }
if ($ApiKey) { $attempts += ,@('shared key', @{ 'X-API-Key' = $ApiKey }) }
foreach ($attempt in $attempts) {
try {
$resp = Invoke-RestMethod -Uri $ApiUrl -Method Post -Body $json `
-ContentType 'application/json' `
-Headers $attempt[1] `
-TimeoutSec $TimeoutSec -ErrorAction Stop
Log ("RESPONSE ({0}) {1}" -f $attempt[0], ($resp | ConvertTo-Json -Compress -Depth 4))
break
} catch {
Log "ERROR POST with $($attempt[0]) failed: $($_.Exception.Message)"
$failure = Get-ShopDbHttpFailure $_
Write-ShopDbAuthFailure $failure
if ($failure.Code -ne 'CREDENTIAL_INVALID') { break }
}
}
exit 0

130
plugins/endpoints/AGENT-CONTRACT.md
View File

@@ -12,7 +12,8 @@ schema changes, update this file in the same change.
key is a UI-managed, endpoints-only service token minted from Settings ->
Endpoint agent key (section 7). A bootstrap `.env` collector key still works
until the first UI key is generated, then it is switched off (section 7). No
querystring fallback.
querystring fallback. Agents 1.20 and later enroll and then send
`X-Endpoint-Credential` instead of the shared key (section 9).
- Body: `application/json`, a single flat object (NOT an array). One POST per
report per endpoint.
- Identity field: `hostname` (required, case-insensitive).
@@ -578,3 +579,130 @@ oversized payload rejected by the server returns nothing at all.
Timed-out runs do not carry output: the script is still running and still holds
the capture file open, so anything read at that moment would be misleading.
## 9. Per-PC credentials
Each endpoint authenticates with its OWN secret, readable only by SYSTEM on
that PC. The server derives the calling asset from the secret. The shared fleet
key (section 7) keeps working for PCs that have not enrolled until an admin
switches it off, and is refused for any PC that has enrolled. The rollout is in
`docs/PER-PC-CREDENTIALS-ROLLOUT.md`.
### 9.1 Headers
| State | Header | Value |
|---|---|---|
| Not enrolled | `X-API-Key` | the shared endpoints key, as today |
| Enrolled | `X-Endpoint-Credential` | `sdbe_` + 43 chars base64url (32 random bytes) |
Never both on one request. The server stores only the SHA-256 of the full
credential plus its first 12 characters (`credentialprefix`) for display.
### 9.2 Enroll
`POST {baseurl}/api/endpoints/enroll`, authenticated with `X-API-Key` while
`plugin.endpoints.sharedkeyallowed` is true. With the shared key switched off
the route takes no auth, but then only returns a credential for a request an
admin has approved.
```jsonc
{
"hostname": "SHOP-PC-014", // 1-255
"serialnumber": "5CD1234ABC", // 1-128, Win32_BIOS.SerialNumber
"deviceid": null, // HKLM\SOFTWARE\GE\ShopDB DeviceId or null
"enrollnonce": "<43 chars>", // 32-128 [A-Za-z0-9_-], generated once, persisted
"agentversion": "1.20.0"
}
```
| Status | Body | Agent action |
|---|---|---|
| 201 | `{"status":"enrolled","credential":"sdbe_...","assetid":123}` | store, switch headers, no restart |
| 202 | `{"status":"pending","requestid":45,"reason":"serial mismatch"}` | retry every 15 min with jitter |
| 403 `ENROLL_NONCE_MISMATCH` | error envelope | an approved request belongs to a different nonce; keep retrying, an admin must reject and re-approve |
| other | error envelope | back off 1, 5, 15 min (cap) |
Auto-issue happens only when `plugin.endpoints.autoenroll` is true, the shared
key authenticated the call, the hostname matches exactly one computer, the
serial on record matches (trimmed, case-insensitive) and the asset has no active
credential. Otherwise the request is pending with one of the reasons `unknown
hostname`, `duplicate hostname`, `no serial on record`, `serial mismatch`,
`already enrolled`, `awaiting approval`, `nonce changed`. The credential is in
the 201 body exactly once.
### 9.3 Error codes
All use the standard error envelope (`data.error.code`).
| HTTP | Code | Meaning | Agent action |
|---|---|---|---|
| 401 | `CREDENTIAL_INVALID` | unknown or revoked credential | delete credential.bin, fall back to the shared key, wake enrollment |
| 401 | `SHARED_KEY_REFUSED` | shared key sent, but this PC is enrolled or the shared key is switched off | wake enrollment; use the credential |
| 403 | `ASSET_MISMATCH` | the credential's asset differs from the hostname/assetid named in the request | log; do not retry the same request |
Routes that authenticate this way: `POST /api/endpoints/collect` (same handler
as `/api/collector/endpoints`), `GET /api/endpoints/monitors/resolved`,
`GET /api/endpoints/commands/next`, `POST /api/endpoints/commands/<id>/result`,
`GET /api/endpoints/apppackage/...`, `GET /api/endpoints/scriptbody/<id>`,
`POST /api/endpoints/operatorreports`, `GET /api/endpoints/alerts`,
`POST /api/endpoints/alerts/<id>/acknowledge`. With a credential, a missing
hostname is allowed and the asset comes from the credential; an acknowledge with
assetid 0 or absent means the caller. The core collector
(`/api/collector/computers`, `/api/collector/printers`) accepts the same header
for the SYSTEM reporter scripts.
### 9.4 Storage on the endpoint
- `%ProgramData%\ShopDB\EndpointAgent\credential\` is ACLed SYSTEM and
Administrators only.
- `credential.bin` = DPAPI `LocalMachine` of UTF-8 JSON
`{"credential":"sdbe_...","assetid":123,"issuedat":"...","baseurl":"..."}`.
SYSTEM scripts read it with
`[System.Security.Cryptography.ProtectedData]::Unprotect(bytes, $null, 'LocalMachine')`
and must not send the credential to a URL outside `baseurl`.
- `enroll.json` = `{"enrollnonce":"...","createdat":"..."}`, plain, same folder.
- The shared key in the registry and appsettings.json is never deleted by the
agent; clearing it is a cutover step in the runbook.
### 9.5 Server pins
- `HKLM\SOFTWARE\GE\ShopDB` `ServerPins` (REG_MULTI_SZ, both registry views)
merged with `serverpins` (string array) in appsettings.json. Legacy
`servercertthumbprint` is still honoured.
- A pin is the lowercase hex SHA-256 of the server leaf certificate's
SubjectPublicKeyInfo. The Server Certificate settings page shows it
(`spkisha256` on `GET /api/endpoints/settings/server-cert`);
`client/Get-ShopDBServerPin.ps1` computes it from a URL or a file;
`client/Set-ShopDBServerPins.ps1` writes it.
- Any pin configured: accept the handshake if and only if the leaf key hash is
in the set (chain and name errors ignored). No pin: require a clean Windows
validation.
### 9.6 Tray pipe
The tray holds no secret and talks to the SYSTEM agent over
`\\.\pipe\ShopDB.EndpointAgent.v1` (SYSTEM and Administrators full control,
INTERACTIVE read/write, 8 instances). One UTF-8 JSON object per line, request
then response, then close; request max 64 KB, read timeout 10 s.
| Request | Response |
|---|---|
| `{"op":"ping"}` | `{"ok":true,"agentversion":"1.20.0","enrolled":true,"assetid":123}` |
| `{"op":"sendstatus"}` | `{"ok":true}` or `{"ok":false,"error":"..."}` |
| `{"op":"sendreport","spoolpath":"C:\\Users\\<user>\\...\\report-xxxx"}` | `{"ok":true}` or `{"ok":false,"error":"..."}`; the path must be inside the caller's own profile and not a reparse point |
| `{"op":"alerts"}` | `{"ok":true,"alerts":[ ...same objects as GET /alerts... ]}` |
| `{"op":"ackalert","notificationid":42}` | `{"ok":true}` or `{"ok":false,"error":"..."}` |
The caller is identified from the pipe (impersonation user name and SID), never
from the request body.
### 9.7 Admin API (WebUI side, for reference)
Read needs `endpoints.fleet`; writes need the admin role and are audited.
- `GET /api/endpoints/enrollment` (`page`, `perpage` <= 100, `state`, `search`)
- `POST /api/endpoints/enrollment/requests/<id>/approve` `{"assetid": n}`
- `POST /api/endpoints/enrollment/requests/<id>/reject`
- `POST /api/endpoints/enrollment/assets/<assetid>/revoke` (un-enroll)
- `PUT /api/endpoints/enrollment/settings` `{"sharedkeyallowed": bool, "autoenroll": bool}`
- `GET /api/endpoints/enrollment/migration` (`page`, `perpage` <= 100, `state`, `search`)
- `POST /api/endpoints/enrollment/migration/wave` `{"assetids": [...]}` or `{"next": n}` (n <= 100)

301
plugins/endpoints/api/templates/upgrade-shopdbagent.ps1
View File

@@ -25,6 +25,14 @@
# succeeds, the agent that would have reported it has been restarted. The new
# version is confirmed on the next check-in. This script's own transcript is
# the local record; see $LogFile below.
#
# AUTH AND PINS
# The download authenticates the way the agent does: the per-PC credential from
# the agent's SYSTEM-only credential.bin when this PC has enrolled, otherwise
# the shared CollectorKey. When ServerPins is set (registry, or serverpins in
# appsettings.json) the server's TLS key must match a pin or nothing is
# downloaded. A 1.19 agent upgrading to 1.20 has no credential yet and uses the
# shared key; the new agent enrolls itself after it starts.
[CmdletBinding()]
param(
@@ -59,6 +67,230 @@ function Write-Log {
try { Add-Content -Path $LogFile -Value $line -ErrorAction Stop } catch { }
}
# The shared credential block below logs through Log, as the reporters do.
function Log([string] $msg) { Write-Log $msg }
# ---------------------------------------------------------------------------
# Per-PC credential and server pin. Kept in step with the same block in
# Report-AssetToShopDB.ps1, Report-PrintersToShopDB.ps1 and
# upgrade-shopdbagent.ps1: each runs standalone, so the text is duplicated.
#
# Credential: the endpoint agent (1.20+) enrolls and stores its own credential
# in credential.bin, DPAPI LocalMachine, readable only by SYSTEM and
# Administrators. When present it is sent as X-Endpoint-Credential and the
# shared key is not sent. Without it the shared CollectorKey goes as X-API-Key.
#
# Pins: HKLM\SOFTWARE\GE\ShopDB ServerPins (REG_MULTI_SZ), lowercase hex
# SHA-256 of the server leaf certificate's SubjectPublicKeyInfo. When any pin
# is set the TLS handshake is accepted only if the leaf key matches one.
# ---------------------------------------------------------------------------
$ShopDbCredentialFile = Join-Path $env:ProgramData 'ShopDB\EndpointAgent\credential\credential.bin'
$ShopDbPinSource = @'
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
public static class ShopDbServerPinValidator
{
private static readonly HashSet<string> Pins = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
public static string LastPresented;
public static void Install(string[] pins)
{
Pins.Clear();
foreach (string pin in pins)
{
if (!String.IsNullOrEmpty(pin)) { Pins.Add(pin); }
}
ServicePointManager.ServerCertificateValidationCallback = Validate;
}
public static bool Validate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors errors)
{
if (Pins.Count == 0) { return errors == SslPolicyErrors.None; }
if (certificate == null) { return false; }
try
{
LastPresented = SpkiSha256Hex(certificate.GetRawCertData());
}
catch (Exception)
{
LastPresented = null;
return false;
}
return Pins.Contains(LastPresented);
}
// Walks the DER: Certificate > TBSCertificate > [0] version (optional),
// serialNumber, signature, issuer, validity, subject, subjectPublicKeyInfo.
public static string SpkiSha256Hex(byte[] der)
{
int length;
int position = EnterSequence(der, 0, out length);
position = EnterSequence(der, position, out length);
if (der[position] == 0xA0) { position = Skip(der, position); }
for (int field = 0; field < 5; field++) { position = Skip(der, position); }
if (der[position] != 0x30) { throw new FormatException("SubjectPublicKeyInfo not found"); }
int end = Skip(der, position);
using (SHA256 sha = SHA256.Create())
{
byte[] hash = sha.ComputeHash(der, position, end - position);
return BitConverter.ToString(hash).Replace("-", "").ToLowerInvariant();
}
}
private static int ReadLength(byte[] der, int position, out int length)
{
int first = der[position];
if (first < 0x80) { length = first; return position + 1; }
int count = first & 0x7F;
if (count == 0 || count > 4) { throw new FormatException("Unsupported DER length"); }
length = 0;
for (int i = 1; i <= count; i++) { length = (length << 8) | der[position + i]; }
return position + 1 + count;
}
private static int EnterSequence(byte[] der, int position, out int length)
{
if (der[position] != 0x30) { throw new FormatException("Expected SEQUENCE"); }
return ReadLength(der, position + 1, out length);
}
private static int Skip(byte[] der, int position)
{
int length;
int content = ReadLength(der, position + 1, out length);
return content + length;
}
}
'@
function Get-ShopDbServerPins {
# Both registry views, like CollectorKey. Anything not 64 hex chars is dropped.
$pins = @()
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64, [Microsoft.Win32.RegistryView]::Registry32)) {
try {
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {
$key = $base.OpenSubKey('SOFTWARE\GE\ShopDB')
if ($key) {
try {
foreach ($value in @($key.GetValue('ServerPins'))) {
if ($null -eq $value) { continue }
$pin = ([string]$value -replace '[\s:]', '').ToLowerInvariant()
if ($pin -match '^[0-9a-f]{64}$' -and $pins -notcontains $pin) { $pins += $pin }
}
} finally { $key.Dispose() }
}
} finally { $base.Dispose() }
} catch {}
}
return ,$pins
}
function Enable-ShopDbServerPins([string[]]$Pins) {
# True when pinning is active. False means pins are set but could not be
# enforced; the caller must not send anything then.
if ($PSVersionTable.PSEdition -eq 'Core') {
# PowerShell 7 web cmdlets ignore ServicePointManager, so the check
# would silently not happen.
Log 'ERROR ServerPins is set; pin checking needs Windows PowerShell 5.1 (powershell.exe), not pwsh'
return $false
}
try {
if (-not ('ShopDbServerPinValidator' -as [type])) {
Add-Type -TypeDefinition $ShopDbPinSource -ErrorAction Stop
}
[ShopDbServerPinValidator]::Install($Pins)
return $true
} catch {
Log "ERROR ServerPins is set but pin checking could not be enabled: $($_.Exception.Message)"
return $false
}
}
function Get-ShopDbEndpointCredential([string]$TargetUrl) {
# Empty when there is none or it cannot be used for this URL.
if (-not (Test-Path -LiteralPath $ShopDbCredentialFile)) { return '' }
try {
Add-Type -AssemblyName System.Security -ErrorAction Stop
$protected = [System.IO.File]::ReadAllBytes($ShopDbCredentialFile)
$plain = [System.Security.Cryptography.ProtectedData]::Unprotect(
$protected, $null, [System.Security.Cryptography.DataProtectionScope]::LocalMachine)
$record = [System.Text.Encoding]::UTF8.GetString($plain) | ConvertFrom-Json
} catch {
Log "WARN could not read the endpoint credential ($ShopDbCredentialFile): $($_.Exception.Message)"
return ''
}
$credential = [string]$record.credential
if (-not $credential.StartsWith('sdbe_')) {
Log 'WARN endpoint credential file has no usable credential; ignoring it'
return ''
}
if (-not $TargetUrl.StartsWith('https://', [System.StringComparison]::OrdinalIgnoreCase)) {
Log "WARN not sending the endpoint credential over a non-https URL ($TargetUrl)"
return ''
}
# Never hand a credential to a server other than the one that issued it.
$issuer = ([string]$record.baseurl).Trim().TrimEnd('/')
if ($issuer -and -not ($TargetUrl.TrimEnd('/') + '/').StartsWith($issuer + '/', [System.StringComparison]::OrdinalIgnoreCase)) {
Log "WARN endpoint credential was issued by $issuer, not the target $TargetUrl; not sending it"
return ''
}
return $credential
}
function Get-ShopDbHttpFailure($ErrorRecord) {
# Status and error envelope code (data.error.code) from a failed request.
$status = 0
$code = ''
$text = ''
try { $status = [int]$ErrorRecord.Exception.Response.StatusCode } catch {}
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
$text = $ErrorRecord.ErrorDetails.Message
} else {
try {
$reader = New-Object System.IO.StreamReader($ErrorRecord.Exception.Response.GetResponseStream())
$text = $reader.ReadToEnd()
$reader.Dispose()
} catch {}
}
if ($text) {
try { $code = [string](($text | ConvertFrom-Json).data.error.code) } catch {}
}
$presented = ''
if ('ShopDbServerPinValidator' -as [type]) { $presented = [string][ShopDbServerPinValidator]::LastPresented }
return @{ Status = $status; Code = $code; PresentedPin = $presented }
}
function Write-ShopDbAuthFailure($Failure) {
switch ($Failure.Code) {
'CREDENTIAL_INVALID' {
Log ('ERROR 401 CREDENTIAL_INVALID: the server does not accept this PC''s endpoint credential ' +
'(revoked or unknown). The endpoint agent deletes it and re-enrolls on its own.')
}
'SHARED_KEY_REFUSED' {
Log ('ERROR 401 SHARED_KEY_REFUSED: the server refuses the shared key for this PC, because ' +
'the PC is enrolled or the shared key is switched off. Reports need the endpoint agent ' +
"credential: check agent 1.20+ is installed, running and enrolled ($ShopDbCredentialFile).")
}
'ASSET_MISMATCH' {
Log ('ERROR 403 ASSET_MISMATCH: this PC''s credential belongs to a different asset than the ' +
'hostname in the report. Check the hostname and the enrollment in ShopDB.')
}
default {
if ($Failure.Status -eq 0 -and $Failure.PresentedPin) {
Log "ERROR TLS refused: server key pin $($Failure.PresentedPin) is not in ServerPins"
}
}
}
}
# Read a value from the 64-bit HKLM view explicitly. If this runs under 32-bit
# PowerShell the HKLM: PSDrive silently redirects to WOW6432Node, where the
# 64-bit agent's config is not, and the upgrade would look unconfigured.
@@ -83,25 +315,33 @@ function Get-AgentConfig {
$baseUrl = Get-Hklm64Value -SubKey 'SOFTWARE\GE\ShopDB' -Name 'BaseUrl'
$apiKey = Get-Hklm64Value -SubKey 'SOFTWARE\GE\ShopDB' -Name 'CollectorKey'
if ([string]::IsNullOrWhiteSpace($baseUrl) -or [string]::IsNullOrWhiteSpace($apiKey)) {
$settings = Join-Path $InstallDir 'appsettings.json'
if (Test-Path $settings) {
try {
$json = Get-Content $settings -Raw | ConvertFrom-Json
$pins = @()
# Read even when the registry is complete: serverpins lives only here.
$settings = Join-Path $InstallDir 'appsettings.json'
if (Test-Path $settings) {
try {
$json = Get-Content $settings -Raw | ConvertFrom-Json
if ([string]::IsNullOrWhiteSpace($baseUrl) -or [string]::IsNullOrWhiteSpace($apiKey)) {
if ([string]::IsNullOrWhiteSpace($baseUrl)) { $baseUrl = $json.baseurl }
if ([string]::IsNullOrWhiteSpace($apiKey)) { $apiKey = $json.apikey }
Write-Log 'read config from appsettings.json (registry unavailable or incomplete)'
}
catch {
Write-Log "could not parse appsettings.json: $($_.Exception.Message)" 'WARN'
foreach ($value in @($json.serverpins)) {
if ($null -eq $value) { continue }
$pin = ([string]$value -replace '[\s:]', '').ToLowerInvariant()
if ($pin -match '^[0-9a-f]{64}$') { $pins += $pin }
}
}
catch {
Write-Log "could not parse appsettings.json: $($_.Exception.Message)" 'WARN'
}
}
if ([string]::IsNullOrWhiteSpace($baseUrl) -or [string]::IsNullOrWhiteSpace($apiKey)) {
throw 'No BaseUrl/CollectorKey found in the registry or appsettings.json. This endpoint does not look installed; run install-shopdbagent.ps1 instead.'
if ([string]::IsNullOrWhiteSpace($baseUrl)) {
throw 'No BaseUrl found in the registry or appsettings.json. This endpoint does not look installed; run install-shopdbagent.ps1 instead.'
}
return @{ BaseUrl = $baseUrl.TrimEnd('/'); ApiKey = $apiKey }
return @{ BaseUrl = $baseUrl.TrimEnd('/'); ApiKey = [string]$apiKey; ServerPins = $pins }
}
# ---------------------------------------------------------------------------
@@ -145,8 +385,45 @@ try {
# TLS 1.2 explicitly: Windows PowerShell 5.1 still defaults to older
# protocols the server will refuse.
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest -Uri $url -Headers @{ 'X-API-Key' = $config.ApiKey } `
-OutFile $zipPath -UseBasicParsing -TimeoutSec 600
$serverPins = @(Get-ShopDbServerPins)
foreach ($pin in $config.ServerPins) {
if ($serverPins -notcontains $pin) { $serverPins += $pin }
}
if ($serverPins.Count -gt 0) {
if (-not (Enable-ShopDbServerPins $serverPins)) {
throw 'ServerPins is set but the pin check could not be enabled; not downloading.'
}
Write-Log "server pin check on ($($serverPins.Count) pin(s))"
}
$credential = Get-ShopDbEndpointCredential $config.BaseUrl
$attempts = @()
if ($credential) { $attempts += ,@('credential', @{ 'X-Endpoint-Credential' = $credential }) }
if ($config.ApiKey) { $attempts += ,@('shared key', @{ 'X-API-Key' = $config.ApiKey }) }
if ($attempts.Count -eq 0) {
throw 'No endpoint credential and no CollectorKey on this endpoint; cannot authenticate the download.'
}
# Credential first. A refused credential (revoked) falls back to the shared
# key once, which the server admits again for an un-enrolled PC.
$downloaded = $false
foreach ($attempt in $attempts) {
try {
Invoke-WebRequest -Uri $url -Headers $attempt[1] `
-OutFile $zipPath -UseBasicParsing -TimeoutSec 600
Write-Log "authenticated with $($attempt[0])"
$downloaded = $true
break
}
catch {
$failure = Get-ShopDbHttpFailure $_
Write-Log "download with $($attempt[0]) failed: $($_.Exception.Message)" 'WARN'
Write-ShopDbAuthFailure $failure
if ($failure.Code -ne 'CREDENTIAL_INVALID') { throw }
}
}
if (-not $downloaded) { throw 'Package download was refused.' }
$size = (Get-Item $zipPath).Length
Write-Log "downloaded $size bytes"

157
plugins/endpoints/client/Get-ShopDBServerPin.ps1 Normal file
View File

@@ -0,0 +1,157 @@
#requires -Version 5.1
<#
.SYNOPSIS
Print the SPKI SHA-256 pin of a ShopDB server certificate, for building the
Set-ShopDBServerPins.ps1 entry.
.DESCRIPTION
The pin is the lowercase hex SHA-256 of the leaf certificate's
SubjectPublicKeyInfo: the same value the Server Certificate settings page
shows and the endpoint agent compares. It depends only on the key pair, so
a certificate renewed with the same key keeps its pin.
Give it either a URL (the certificate the server actually presents is read
from the TLS handshake; it is NOT trusted or validated, only hashed) or a
certificate file (.cer, .crt, .pem, DER or PEM). Use the file form to pin a
new certificate BEFORE it goes live on the server.
Reading from a URL trusts the network path at that moment. Compare the
printed pin with the one on the Server Certificate settings page before
rolling it out.
.PARAMETER Url
https://<server>[:port]. Only the host and port are used.
.PARAMETER Path
A certificate file.
.EXAMPLE
.\Get-ShopDBServerPin.ps1 -Url https://<server>
.EXAMPLE
.\Get-ShopDBServerPin.ps1 -Path .\server.crt
#>
[CmdletBinding(DefaultParameterSetName = 'Url')]
param(
[Parameter(Mandatory = $true, ParameterSetName = 'Url', Position = 0)]
[string]$Url,
[Parameter(Mandatory = $true, ParameterSetName = 'Path')]
[string]$Path,
[int]$TimeoutSec = 15
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
$source = @'
using System;
using System.IO;
using System.Net.Security;
using System.Net.Sockets;
using System.Security.Authentication;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
public static class ShopDbServerPinReader
{
public static byte[] FetchLeafCertificate(string host, int port, int timeoutMilliseconds)
{
using (TcpClient client = new TcpClient())
{
IAsyncResult pending = client.BeginConnect(host, port, null, null);
if (!pending.AsyncWaitHandle.WaitOne(timeoutMilliseconds))
{
throw new TimeoutException("Timed out connecting to " + host + ":" + port);
}
client.EndConnect(pending);
client.ReceiveTimeout = timeoutMilliseconds;
client.SendTimeout = timeoutMilliseconds;
byte[] captured = null;
RemoteCertificateValidationCallback capture = delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors errors)
{
if (certificate != null) { captured = certificate.GetRawCertData(); }
return true;
};
using (SslStream ssl = new SslStream(client.GetStream(), false, capture))
{
ssl.AuthenticateAsClient(host, null, SslProtocols.Tls12, false);
}
if (captured == null) { throw new InvalidOperationException("The server presented no certificate"); }
return captured;
}
}
// Walks the DER: Certificate > TBSCertificate > [0] version (optional),
// serialNumber, signature, issuer, validity, subject, subjectPublicKeyInfo.
public static string SpkiSha256Hex(byte[] der)
{
int length;
int position = EnterSequence(der, 0, out length);
position = EnterSequence(der, position, out length);
if (der[position] == 0xA0) { position = Skip(der, position); }
for (int field = 0; field < 5; field++) { position = Skip(der, position); }
if (der[position] != 0x30) { throw new FormatException("SubjectPublicKeyInfo not found"); }
int end = Skip(der, position);
using (SHA256 sha = SHA256.Create())
{
byte[] hash = sha.ComputeHash(der, position, end - position);
return BitConverter.ToString(hash).Replace("-", "").ToLowerInvariant();
}
}
private static int ReadLength(byte[] der, int position, out int length)
{
int first = der[position];
if (first < 0x80) { length = first; return position + 1; }
int count = first & 0x7F;
if (count == 0 || count > 4) { throw new FormatException("Unsupported DER length"); }
length = 0;
for (int i = 1; i <= count; i++) { length = (length << 8) | der[position + i]; }
return position + 1 + count;
}
private static int EnterSequence(byte[] der, int position, out int length)
{
if (der[position] != 0x30) { throw new FormatException("Expected SEQUENCE"); }
return ReadLength(der, position + 1, out length);
}
private static int Skip(byte[] der, int position)
{
int length;
int content = ReadLength(der, position + 1, out length);
return content + length;
}
}
'@
if (-not ('ShopDbServerPinReader' -as [type])) {
Add-Type -TypeDefinition $source
}
try {
if ($PSCmdlet.ParameterSetName -eq 'Path') {
$resolved = (Resolve-Path -LiteralPath $Path).ProviderPath
$certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $resolved
$der = $certificate.RawData
$label = "$resolved (subject $($certificate.Subject), expires $($certificate.NotAfter.ToString('yyyy-MM-dd')))"
} else {
$uri = [Uri]$Url
if ($uri.Scheme -ne 'https') { throw "The URL must be https ($Url)" }
$der = [ShopDbServerPinReader]::FetchLeafCertificate($uri.Host, $uri.Port, $TimeoutSec * 1000)
$certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList (,$der)
$label = "$($uri.Host):$($uri.Port) (subject $($certificate.Subject), expires $($certificate.NotAfter.ToString('yyyy-MM-dd')))"
}
$pin = [ShopDbServerPinReader]::SpkiSha256Hex($der)
} catch {
Write-Error "Could not read a certificate: $($_.Exception.Message)"
exit 1
}
Write-Host "Certificate: $label"
Write-Host "Pin (SPKI SHA-256):"
Write-Output $pin
exit 0

182
plugins/endpoints/client/Set-ShopDBServerPins.ps1 Normal file
View File

@@ -0,0 +1,182 @@
#requires -Version 5.1
<#
.SYNOPSIS
Write the ShopDB server public key pins to HKLM:\SOFTWARE\GE\ShopDB
ServerPins, so the endpoint agent and the SYSTEM reporters trust only this
site's server key.
.DESCRIPTION
Each pin is the lowercase hex SHA-256 of the server leaf certificate's
SubjectPublicKeyInfo. The ShopDB Server Certificate settings page shows the
current one, and Get-ShopDBServerPin.ps1 computes one from a URL or a
certificate file.
Written as REG_MULTI_SZ in BOTH registry views (64-bit and WOW6432Node),
the same way CollectorKey is, so a 32-bit reader sees the same set.
Meant to run as a GE-Enforce PS1 entry under the SYSTEM task, before agent
1.20 reaches the PC. Idempotent: the set on the PC is compared with the
requested set first, and nothing is written when they already match, so it
is safe every cycle.
ROTATING THE SERVER KEY: pass the old pin AND the new pin until every PC has
both, switch the server certificate, then drop the old pin. A PC holding
only the old pin refuses the new certificate.
.PARAMETER Pins
One or more pins separated by commas, semicolons or spaces. Colons and case
are ignored. Every value must be 64 hex characters or nothing is written.
.PARAMETER Clear
Remove ServerPins from both views. Pinning is then off and the normal
Windows certificate checks apply.
.PARAMETER LogPath
Append a transcript here as well as writing to the pipeline.
.EXAMPLE
.\Set-ShopDBServerPins.ps1 -Pins <pin>
Pin one server key.
.EXAMPLE
.\Set-ShopDBServerPins.ps1 -Pins "<oldpin>,<newpin>" -WhatIf
Show what a key rotation would write without touching the registry.
.NOTES
Exit codes:
0 written, or already correct
1 bad input or an error stopped the write
#>
[CmdletBinding(SupportsShouldProcess = $true)]
param(
[string]$Pins,
[switch]$Clear,
[string]$LogPath
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
$SubKey = 'SOFTWARE\GE\ShopDB'
$ValueName = 'ServerPins'
function Log {
param([string]$Message)
$line = "{0} [Set-ShopDBServerPins] {1}" -f (Get-Date -Format 'yyyy-MM-dd HH:mm:ss'), $Message
Write-Output $line
if ($LogPath) {
try { Add-Content -LiteralPath $LogPath -Value $line -ErrorAction Stop } catch { }
}
}
function Get-CurrentPins {
param([Microsoft.Win32.RegistryView]$View)
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine, $View)
try {
$key = $base.OpenSubKey($SubKey)
if (-not $key) { return @() }
try {
$value = $key.GetValue($ValueName)
if ($null -eq $value) { return @() }
return @(@($value) | ForEach-Object { ([string]$_).Trim().ToLowerInvariant() } | Where-Object { $_ })
} finally { $key.Dispose() }
} finally { $base.Dispose() }
}
function Test-SamePins {
param([string[]]$Left, [string[]]$Right)
$a = @($Left | Sort-Object -Unique)
$b = @($Right | Sort-Object -Unique)
if ($a.Count -ne $b.Count) { return $false }
for ($i = 0; $i -lt $a.Count; $i++) {
if ($a[$i] -ne $b[$i]) { return $false }
}
return $true
}
# --- 1. validate the request ------------------------------------------------
$requested = @()
if (-not $Clear) {
if (-not $Pins -or -not $Pins.Trim()) {
Log 'ERROR no -Pins given (use -Clear to remove pinning)'
exit 1
}
foreach ($raw in ($Pins -split '[,;\s]+')) {
if (-not $raw) { continue }
$pin = ($raw -replace ':', '').ToLowerInvariant()
if ($pin -notmatch '^[0-9a-f]{64}$') {
Log "ERROR '$raw' is not a SHA-256 pin (64 hex characters); nothing written"
exit 1
}
if ($requested -notcontains $pin) { $requested += $pin }
}
if ($requested.Count -eq 0) {
Log 'ERROR -Pins held no values; nothing written'
exit 1
}
Log "requested $($requested.Count) pin(s): $($requested -join ', ')"
}
# --- 2. write each view -----------------------------------------------------
$views = @(
@{ Name = '64-bit'; View = [Microsoft.Win32.RegistryView]::Registry64 },
@{ Name = '32-bit'; View = [Microsoft.Win32.RegistryView]::Registry32 }
)
$failed = $false
foreach ($entry in $views) {
try {
$current = @(Get-CurrentPins -View $entry.View)
if ($Clear) {
if ($current.Count -eq 0) {
Log "$($entry.Name): no ServerPins; nothing to do"
continue
}
if ($PSCmdlet.ShouldProcess("HKLM\$SubKey ($($entry.Name))", "Remove $ValueName")) {
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine, $entry.View)
try {
$key = $base.OpenSubKey($SubKey, $true)
if ($key) {
try { $key.DeleteValue($ValueName, $false) } finally { $key.Dispose() }
}
} finally { $base.Dispose() }
Log "$($entry.Name): removed ServerPins"
}
continue
}
if (Test-SamePins -Left $current -Right $requested) {
Log "$($entry.Name): ServerPins already correct; nothing to do"
continue
}
$action = "Set $ValueName to $($requested.Count) pin(s) (was $($current.Count))"
if ($PSCmdlet.ShouldProcess("HKLM\$SubKey ($($entry.Name))", $action)) {
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine, $entry.View)
try {
$key = $base.CreateSubKey($SubKey)
try {
$key.SetValue($ValueName, [string[]]$requested, [Microsoft.Win32.RegistryValueKind]::MultiString)
} finally { $key.Dispose() }
} finally { $base.Dispose() }
# Read back rather than trust SetValue.
$after = @(Get-CurrentPins -View $entry.View)
if (-not (Test-SamePins -Left $after -Right $requested)) {
Log "ERROR $($entry.Name): ServerPins did not read back as written"
$failed = $true
continue
}
Log "$($entry.Name): wrote ServerPins ($($requested.Count) pin(s), was $($current.Count))"
}
} catch {
Log "ERROR $($entry.Name): $($_.Exception.Message)"
$failed = $true
}
}
if ($failed) { exit 1 }
exit 0

238
plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md Normal file
View File

@@ -0,0 +1,238 @@
# Per-PC credentials rollout
Runbook for moving a site's endpoint fleet from the shared collector key to
per-PC credentials. The wire contract is `AGENT-CONTRACT.md` section 9.
Placeholders: `<server>` is the ShopDB base URL, `<share>` the GE-Enforce share
root, `<pin>` a server key pin, `<pilotbay>` / `<pilothost>` the pilot PC.
## What changes
- Every endpoint gets its own secret (`sdbe_...`), stored in
`%ProgramData%\ShopDB\EndpointAgent\credential\credential.bin`, DPAPI
LocalMachine, readable by SYSTEM and Administrators only.
- The server knows which asset is calling from the secret. A PC can no longer
report as another PC.
- The tray holds no secret. It talks to the SYSTEM agent over a local pipe.
- The agent, the upgrade script and the SYSTEM reporters
(`Report-AssetToShopDB.ps1`, `Report-PrintersToShopDB.ps1`) pin the server's
public key when `ServerPins` is set.
- The shared key keeps working for PCs that have NOT enrolled, until an admin
turns it off. An enrolled PC is always refused the shared key
(`401 SHARED_KEY_REFUSED`).
## Before you start
1. Server deployed with the endpoints migration `endpoints0023credentials`
applied (`flask plugin upgrade-all`).
2. Settings -> Endpoints -> Server Certificate shows a **Public key pin (SPKI
SHA-256)**. Cross-check it from a PC:
`Get-ShopDBServerPin.ps1 -Url https://<server>`. The two must match.
3. Agent 1.20 (and tray 1.7) uploaded as the current release under Settings ->
Endpoints -> Upload New/Legacy Agents. Keep the previous release staged as a
legacy build: it is the rollback.
4. Settings -> Endpoints -> Enrollment -> Credentials: **Shared key allowed**
on, **Auto-enroll** on (both are the defaults).
5. The GE-Enforce manifest scope that will carry the entries below is at
`Version` 2.2 or later, and the fleet's GE-Enforce client library
understands `TargetHostnames` / `TargetMachineNumbers`. An older manifest or
library ignores those lists and applies the entry to EVERY PC in scope. The
same happens with an empty list: never publish a targeted entry with a blank
target.
## Phase 1: pins on the PCs
Pins must be on a PC BEFORE agent 1.20 arrives there. A 1.20 agent with no pin
falls back to plain Windows validation, which a self-signed or internal-CA
server may fail; a pin set after the fact is not a rollout order you want to
debug per bay.
Stage `plugins/endpoints/client/Set-ShopDBServerPins.ps1` on the share and add
a PS1 entry. Pilot first:
```json
{
"_comment": "Server key pin for per-PC credentials. Pilot only.",
"Name": "ShopDB server pins",
"Type": "PS1",
"Script": "scripts/Set-ShopDBServerPins.ps1",
"Args": "-Pins <pin>",
"DetectionMethod": "Always",
"TargetMachineNumbers": ["<pilotbay>"]
}
```
Then widen by hostname list (or machine number list) in batches:
```json
{
"Name": "ShopDB server pins",
"Type": "PS1",
"Script": "scripts/Set-ShopDBServerPins.ps1",
"Args": "-Pins <pin>",
"DetectionMethod": "Always",
"TargetHostnames": ["<host1>", "<host2>", "<host3>"]
}
```
The script compares before it writes, so `Always` costs nothing per cycle.
**Gate:** on the pilot, `HKLM\SOFTWARE\GE\ShopDB` and
`HKLM\SOFTWARE\WOW6432Node\GE\ShopDB` both hold `ServerPins` (REG_MULTI_SZ)
equal to the pin on the settings page, and the next `report-asset-*.log` under
`C:\Logs\Shopfloor` says `server pin check on (1 pin(s))` followed by a
RESPONSE line.
**Rotating the server key later:** put the old AND the new pin in `-Pins`
(`"<oldpin>,<newpin>"`), wait until every PC has both, switch the certificate,
then drop the old pin.
## Phase 2: re-stamp the share's agent entries
PCs that got the agent from the share (collections manifest entries with a
`Hash` detection on `shopdbagent.exe`) are held to that hash by GE-Enforce. If
a store upgrade wave puts 1.20 on such a PC while the entry still carries the
1.19 hash, GE-Enforce reinstalls 1.19 on its next cycle and the PC flips back.
Before any wave reaches a share-installed PC, either:
- re-stamp those entries to the 1.20 payload and its `DetectionValue` SHA-256
(and keep the `ProductCode`/version guards the entries already have), or
- remove the entries for the PCs the wave will cover.
Agent entries stay targeted. Illustrative shape for the pilot (keep your
existing entry's type, installer and guards; what matters is the hash and a
non-empty target):
```json
{
"Name": "ShopDB endpoint agent 1.20",
"Type": "PS1",
"Script": "apps/shopdbagent/1.20.0/install-shopdbagent.ps1",
"DetectionMethod": "Hash",
"DetectionPath": "C:\\Program Files\\ShopDB\\EndpointAgent\\shopdbagent.exe",
"DetectionValue": "<sha256 of the 1.20 shopdbagent.exe>",
"TargetHostnames": ["<pilothost>"]
}
```
**Gate:** the manifest preview for the pilot shows the 1.20 hash, and no entry
for the same file with a blank target list exists in any scope that reaches it.
## Phase 3: pilot one PC
1. Enrollment -> Migration: find `<pilothost>`. It should be **Ready**. If it is
**Blocked**, the Blockers column lists every reason; fix those first.
2. Tick it, **Upgrade selected**.
3. Watch the row: **Upgrading**, then **Enrolled** within a few minutes of the
new agent starting. The agent enrolls on its own with the shared key it
already holds.
**Gate before any wave:**
- Credentials tab shows the PC enrolled, `enrolledvia` = `auto`, and a
`lastusedat` that keeps moving.
- The PC's Fleet row keeps reporting; commands (Collect now) still work.
- The tray shows alerts and Send Status works (it now goes through the pipe).
- `report-asset-*.log` and `report-printers-*.log` show `RESPONSE (credential)`.
- Nothing reinstalled the old agent after two GE-Enforce cycles.
## Phase 4: waves
Migration tab -> **Upgrade next 25**. The confirm dialog states how many PCs get
agent 1.20. The server re-checks readiness per PC and packages the agent once
per wave; PCs that stopped being ready are returned as skipped.
Between waves, in the Waves list:
- wait until the wave shows **Done** (every PC enrolled or failed),
- investigate every **failed** PC before the next wave,
- check nothing moved from Enrolled back to Upgrading or Ready (a share entry
reinstalling the old agent, see phase 2).
Repeat until **Ready** is 0.
## Phase 5: stragglers and new PCs
- **Blocked:** each blocker names its fix. `offline`: bring the PC back or wait.
`no serial on record` / `duplicate hostname`: fix the computer record.
`commands locked`: release the endpoint lock or policy hold. `no agent package
in store`: upload 1.20. `agent too old for remote upgrade`: move it via the
share (targeted agent entry as in phase 2).
- **Not reporting:** the PC has no working agent. It can only move via the
share channel: add its hostname to a targeted 1.20 agent entry. It shows as
Pending or Enrolled once the agent runs.
- **Pending:** the agent asked but auto-enroll could not decide (serial
mismatch, duplicate or unknown hostname, already enrolled, or auto-enroll
off). Credentials tab -> Requests -> **Approve**; the picker defaults to the
matched computer, search to choose another. The PC collects its credential on
its next retry (within 15 minutes). **Reject** anything you do not recognise.
- **Newly imaged PCs:** imaging installs 1.20 with the shared key, and the agent
auto-enrolls on first start. After the shared key is turned off (phase 6) a
new PC cannot auto-enroll: its request lands in Pending and an admin approves
it in the UI. That is the steady-state path for new PCs.
## Phase 6: cutover
**Gate:** Credentials tab **Not enrolled** is 0, or every remaining PC is one
you have decided may stop reporting. Remember that turning the shared key off
also refuses it on `/api/collector/computers` and `/api/collector/printers`, so
a PC with reporter scripts but no enrolled agent goes silent.
1. Turn **Shared key allowed** off. The confirm shows how many PCs are not
enrolled; type `OFF`.
2. Watch Fleet for an hour. A PC that went quiet is either unenrolled (turn the
shared key back on, fix it, try again) or failed pinning.
3. Remove the shared key from every place it lives (below).
4. Rotate the old key so any copy you missed is dead: Settings -> Endpoints ->
**Regenerate key** (and rotate the collector token the reporters used, if it
was a separate token). With the shared key off nothing on the floor needs
the new key: approved enroll requests are collected without it. Store it
the way the key page describes and keep it out of every file below. A
fleet rollback after this point (see Rollback) needs the new key pushed to
the PCs again.
### Every place the shared key lives
| Where | Put there by | How to clear at cutover |
|---|---|---|
| GE-Enforce manifest entry `Args` (`-ApiKey ...`) | whoever authored the entry | Remove the argument from the entry and publish. Search every scope for the key prefix. |
| `HKLM\SOFTWARE\GE\ShopDB` `CollectorKey`, both registry views | agent `install.ps1`; `Set-ShopdbCollectorKey.ps1` from `<share>\common\configs\shopdb-collector.txt` | First retire the `Set-ShopdbCollectorKey.ps1` entry (or it rewrites the value every cycle), then a targeted PS1 entry that deletes `CollectorKey` in both views. |
| `<share>\common\configs\shopdb-collector.txt` | share staging | Delete the file after the entry above is retired. |
| `appsettings.json` `apikey` in the agent install folder | agent package / hand edits | Targeted PS1 entry that sets `apikey` to an empty string. The agent never needs it once enrolled. |
| Generated `install.ps1` default `-CollectorKey` (Build site agent downloads) | Settings -> Endpoints -> Build site agent | Delete every kept copy (share, USB, work folders). Rebuild after rotation only if you still need the package. |
| Display kiosk `shopdb-key.txt` | display imaging | Confirm what the kiosk uses it for before clearing; a kiosk has no endpoint agent, so anything it reports with the shared key stops at cutover. Delete the file once that is settled. |
The agent never deletes the shared key itself: other SYSTEM scripts may still
read it until this table is done.
## Rollback
**One PC:**
1. Make sure **Shared key allowed** is on.
2. Credentials tab -> **Un-enroll** the PC. Its credential stops working
immediately; the server admits the shared key for it again.
3. Push the previous agent: Upload New/Legacy Agents -> make the legacy build
current and upgrade that PC, or re-stamp its share entry to the old hash.
4. Remove the PC from the pin entry's targets only if pinning itself is the
problem (`Set-ShopDBServerPins.ps1 -Clear` removes the value).
A 1.20 agent that is un-enrolled but NOT downgraded simply enrolls again on its
next cycle; that is expected.
**Whole fleet:** turn **Shared key allowed** back on first. Every PC that still
holds the shared key reports again immediately. Then un-enroll and downgrade
PCs as above, in waves.
## Quick reference
- Pin of the live server: Server Certificate settings page, or
`Get-ShopDBServerPin.ps1 -Url https://<server>`.
- Pin of a certificate file: `Get-ShopDBServerPin.ps1 -Path <file>`.
- Agent log of enrollment: `%ProgramData%\ShopDB\EndpointAgent\logs`.
- Reporter logs: `C:\Logs\Shopfloor\report-asset-*.log`,
`report-printers-*.log`. `SHARED_KEY_REFUSED` there means the PC is enrolled
or the shared key is off and the reporter could not read the credential;
`CREDENTIAL_INVALID` means the PC was un-enrolled and the agent is
re-enrolling.

12
plugins/endpoints/frontend/routes.js
View File

@@ -43,6 +43,18 @@ export default [
requiresPermission: 'endpoints.runscript',
plugin: 'endpoints'
}
},
// Readable by endpoints.fleet holders so they can watch a migration; every
// write on the page is admin-only and re-checked server-side.
{
path: 'settings/endpoints/enrollment',
name: 'endpoints-enrollment-settings',
component: () => import('./views/EnrollmentSettings.vue'),
meta: {
requiresAuth: true,
requiresPermission: 'endpoints.fleet',
plugin: 'endpoints'
}
}
]

989
plugins/endpoints/frontend/views/EnrollmentSettings.vue Normal file
View File

@@ -0,0 +1,989 @@
<template>
<div>
<div class="page-header">
<h2>Enrollment</h2>
</div>
<p class="setting-description">
Each endpoint agent authenticates with its own credential, readable only
by SYSTEM on that PC. Agents from {{ target }} onward enroll themselves
with the shared key they already hold; older agents move over by
upgrading. Use Migration to move the fleet, and Credentials to approve
requests, un-enroll a PC, and retire the shared key.
</p>
<div class="enroll-tabs">
<button
type="button"
class="enroll-tab"
:class="{ active: tab === 'migration' }"
@click="tab = 'migration'"
>
Migration
</button>
<button
type="button"
class="enroll-tab"
:class="{ active: tab === 'credentials' }"
@click="tab = 'credentials'"
>
Credentials
</button>
</div>
<!-- ==================================================================
Migration
=============================================================== -->
<section v-if="tab === 'migration'" class="migration-tab">
<div v-if="migrationError" class="load-error">{{ migrationError }}</div>
<div class="enroll-tiles">
<button
v-for="tile in migrationTiles"
:key="tile.state"
type="button"
class="enroll-tile"
:class="[tile.tone, { on: migrationState === tile.state }]"
@click="toggleMigrationState(tile.state)"
>
<span class="n" :data-count="tile.state">{{ migrationCounts[tile.state] || 0 }}</span>
<span class="l">{{ tile.label }}</span>
</button>
</div>
<div class="progress-row">
<div class="progress-track">
<div class="progress-fill" :style="{ width: enrolledPercent + '%' }"></div>
</div>
<span class="progress-text">
{{ migrationCounts.enrolled || 0 }} of {{ migrationTotal }} enrolled
</span>
</div>
<div class="filters">
<input
v-model="migrationSearch"
type="text"
class="form-control"
placeholder="Search hostname..."
@input="debouncedMigrationSearch"
/>
<select v-model="migrationState" class="form-control enroll-filter" @change="reloadMigration">
<option value="">All states</option>
<option v-for="tile in migrationTiles" :key="tile.state" :value="tile.state">
{{ tile.label }}
</option>
</select>
</div>
<div v-if="isadmin" class="bulk-bar">
<span class="bulk-count">{{ selected.length }} selected</span>
<button
type="button"
class="btn btn-primary btn-sm upgrade-selected"
:disabled="busy || !selected.length"
@click="upgradeSelected"
>
Upgrade selected
</button>
<button
type="button"
class="btn btn-secondary btn-sm upgrade-next"
:disabled="busy || !(migrationCounts.ready > 0)"
@click="upgradeNext"
>
Upgrade next {{ waveSize }}
</button>
</div>
<div class="card">
<div v-if="migrationLoading" class="loading">Loading...</div>
<div v-else class="table-container">
<table>
<thead>
<tr>
<th v-if="isadmin" class="select-cell">
<input
type="checkbox"
:checked="allReadySelected"
:disabled="!readyRows.length"
title="Select every ready PC on this page"
@change="toggleAllReady"
/>
</th>
<th>Hostname</th>
<th>State</th>
<th>Agent version</th>
<th>Last reported</th>
<th>Blockers</th>
</tr>
</thead>
<tbody>
<tr v-if="!migrationRows.length">
<td :colspan="isadmin ? 6 : 5" class="empty-cell">No computers match.</td>
</tr>
<tr v-for="row in migrationRows" :key="row.assetid" class="migration-row">
<td v-if="isadmin" class="select-cell">
<input
v-if="row.state === 'ready'"
v-model="selected"
type="checkbox"
:value="row.assetid"
/>
</td>
<td>{{ row.hostname || '-' }}</td>
<td><span class="state-chip" :class="row.state">{{ stateLabel(row.state) }}</span></td>
<td>{{ row.agentversion || '-' }}</td>
<td>{{ formatDate(row.lastreportedat) }}</td>
<td class="blockers">{{ (row.blockers || []).join(', ') || '-' }}</td>
</tr>
</tbody>
</table>
</div>
<PaginationBar
:page="migrationPage"
:total-pages="migrationTotalPages"
@update:page="goToMigrationPage"
/>
</div>
<h3 class="section-title">Waves</h3>
<div class="card">
<div class="table-container">
<table>
<thead>
<tr>
<th>Wave</th>
<th>Created</th>
<th>By</th>
<th>Size</th>
<th>Queued</th>
<th>Enrolled</th>
<th>Failed</th>
<th>Progress</th>
</tr>
</thead>
<tbody>
<tr v-if="!waves.length">
<td colspan="8" class="empty-cell">No waves yet.</td>
</tr>
<tr v-for="wave in waves" :key="wave.waveid">
<td>{{ wave.waveid }}</td>
<td>{{ formatDate(wave.createdat) }}</td>
<td>{{ wave.createdby || '-' }}</td>
<td>{{ wave.size }}</td>
<td>{{ wave.queued }}</td>
<td>{{ wave.enrolled }}</td>
<td>{{ wave.failed }}</td>
<td>
<div class="progress-track small">
<div class="progress-fill" :style="{ width: wavePercent(wave) + '%' }"></div>
</div>
<span class="field-hint">{{ wave.done ? 'Done' : 'In progress' }}</span>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</section>
<!-- ==================================================================
Credentials
=============================================================== -->
<section v-else class="credentials-tab">
<div v-if="enrollmentError" class="load-error">{{ enrollmentError }}</div>
<div class="card form-card">
<h3>Settings</h3>
<label class="checkbox-row">
<input
type="checkbox"
class="toggle-sharedkey"
:checked="enrollmentSettings.sharedkeyallowed"
:disabled="!isadmin || busy"
@change="onSharedKeyToggle($event)"
/>
<span>
<strong>Shared key allowed.</strong> PCs that have not enrolled may
still report with the fleet shared key. An enrolled PC is always
refused the shared key. Turn this off once every PC is enrolled.
</span>
</label>
<label class="checkbox-row">
<input
type="checkbox"
class="toggle-autoenroll"
:checked="enrollmentSettings.autoenroll"
:disabled="!isadmin || busy"
@change="onAutoEnrollToggle($event)"
/>
<span>
<strong>Auto-enroll.</strong> Issue a credential without approval
when the hostname matches exactly one computer and the serial number
on record matches. Anything else waits below for an admin.
</span>
</label>
</div>
<div class="enroll-tiles">
<div class="enroll-tile static ok">
<span class="n" data-count="enrolled">{{ enrollmentCounts.enrolled || 0 }}</span>
<span class="l">Enrolled</span>
</div>
<div class="enroll-tile static warn">
<span class="n" data-count="notenrolled">{{ enrollmentCounts.notenrolled || 0 }}</span>
<span class="l">Not enrolled</span>
</div>
<div class="enroll-tile static">
<span class="n" data-count="pending">{{ enrollmentCounts.pending || 0 }}</span>
<span class="l">Pending requests</span>
</div>
</div>
<h3 class="section-title">Requests</h3>
<div class="card">
<div class="table-container">
<table>
<thead>
<tr>
<th>Hostname</th>
<th>Serial</th>
<th>Device id</th>
<th>Reason</th>
<th>Status</th>
<th>Source</th>
<th>Last seen</th>
<th v-if="isadmin"></th>
</tr>
</thead>
<tbody>
<tr v-if="!requests.length">
<td :colspan="isadmin ? 8 : 7" class="empty-cell">No enrollment requests.</td>
</tr>
<template v-for="request in requests" :key="request.requestid">
<tr class="request-row">
<td>{{ request.hostname }}</td>
<td>{{ request.serialnumber || '-' }}</td>
<td>{{ request.deviceid || '-' }}</td>
<td>{{ request.reason || '-' }}</td>
<td>{{ request.status }}</td>
<td>{{ request.sourceip || '-' }}</td>
<td>{{ formatDate(request.lastseenat) }}</td>
<td v-if="isadmin" class="action-cell">
<template v-if="request.status === 'pending'">
<button
type="button"
class="btn btn-primary btn-sm approve-open"
:disabled="busy"
@click="openApprove(request)"
>
Approve
</button>
<button
type="button"
class="btn btn-secondary btn-sm reject"
:disabled="busy"
@click="reject(request)"
>
Reject
</button>
</template>
</td>
</tr>
<tr v-if="approving && approving.requestid === request.requestid" class="approve-row">
<td :colspan="isadmin ? 8 : 7">
<div class="approve-form">
<label class="field-label">Issue the credential for computer</label>
<input
v-model="approving.search"
type="text"
class="form-control asset-search"
placeholder="Search hostname or serial..."
@input="debouncedAssetSearch"
/>
<select v-model="approving.assetid" class="form-control asset-picker">
<option :value="null">Choose a computer</option>
<option
v-for="option in approving.options"
:key="option.assetid"
:value="option.assetid"
>
{{ option.label }}
</option>
</select>
<button
type="button"
class="btn btn-primary btn-sm approve-confirm"
:disabled="busy || !approving.assetid"
@click="approve"
>
Approve
</button>
<button type="button" class="btn btn-secondary btn-sm" @click="approving = null">
Cancel
</button>
</div>
<p class="field-hint">
The PC receives its credential the next time it asks,
which is within 15 minutes. Only the PC that made this
request can collect it.
</p>
</td>
</tr>
</template>
</tbody>
</table>
</div>
</div>
<h3 class="section-title">Computers</h3>
<div class="filters">
<input
v-model="enrollmentSearch"
type="text"
class="form-control"
placeholder="Search hostname..."
@input="debouncedEnrollmentSearch"
/>
<select v-model="enrollmentState" class="form-control enroll-filter" @change="reloadEnrollment">
<option value="">All states</option>
<option value="enrolled">Enrolled</option>
<option value="notenrolled">Not enrolled</option>
<option value="pending">Pending</option>
<option value="revoked">Revoked</option>
</select>
</div>
<div class="card">
<div v-if="enrollmentLoading" class="loading">Loading...</div>
<div v-else class="table-container">
<table>
<thead>
<tr>
<th>Hostname</th>
<th>Serial</th>
<th>State</th>
<th>Credential</th>
<th>Issued</th>
<th>Last used</th>
<th>Via</th>
<th v-if="isadmin"></th>
</tr>
</thead>
<tbody>
<tr v-if="!credentialRows.length">
<td :colspan="isadmin ? 8 : 7" class="empty-cell">No computers match.</td>
</tr>
<tr v-for="row in credentialRows" :key="row.assetid">
<td>{{ row.hostname || '-' }}</td>
<td>{{ row.serialnumber || '-' }}</td>
<td><span class="state-chip" :class="row.state">{{ stateLabel(row.state) }}</span></td>
<td>
<code v-if="row.credentialprefix" class="mono">{{ row.credentialprefix }}...</code>
<span v-else>-</span>
</td>
<td>{{ formatDate(row.issuedat) }}</td>
<td>{{ formatDate(row.lastusedat) }}</td>
<td>{{ row.enrolledvia || '-' }}</td>
<td v-if="isadmin" class="action-cell">
<button
v-if="row.state === 'enrolled'"
type="button"
class="btn btn-danger btn-sm revoke"
:disabled="busy"
@click="revoke(row)"
>
Un-enroll
</button>
</td>
</tr>
</tbody>
</table>
</div>
<PaginationBar
:page="enrollmentPage"
:total-pages="enrollmentTotalPages"
@update:page="goToEnrollmentPage"
/>
</div>
</section>
</div>
</template>
<script setup>
import { ref, computed, watch, onMounted, onBeforeUnmount } from 'vue'
import api, { computersApi } from '@/api'
import { useAuthStore } from '@/stores/auth'
import { useToast } from '@/composables/toast'
import { apiError } from '@/utils/apiError'
import PaginationBar from '@/components/PaginationBar.vue'
const PERPAGE = 50
const POLL_MS = 30000
const authStore = useAuthStore()
const toast = useToast()
// UI convenience only; every write is re-checked server-side.
const isadmin = computed(() => authStore.isAdmin)
const tab = ref('migration')
const busy = ref(false)
const waveSize = 25
// ---- Migration --------------------------------------------------------
const migrationTiles = [
{ state: 'enrolled', label: 'Enrolled', tone: 'ok' },
{ state: 'ready', label: 'Ready', tone: 'ok' },
{ state: 'blocked', label: 'Blocked', tone: 'bad' },
{ state: 'upgrading', label: 'Upgrading', tone: '' },
{ state: 'pending', label: 'Pending', tone: 'warn' },
{ state: 'notreporting', label: 'Not reporting', tone: '' }
]
const target = ref('1.20.0')
const migrationCounts = ref({})
const migrationRows = ref([])
const waves = ref([])
const migrationPage = ref(1)
const migrationTotalPages = ref(1)
const migrationState = ref('')
const migrationSearch = ref('')
const migrationLoading = ref(false)
const migrationError = ref('')
const selected = ref([])
const migrationTotal = computed(() =>
migrationTiles.reduce((sum, tile) => sum + (Number(migrationCounts.value[tile.state]) || 0), 0))
const enrolledPercent = computed(() => {
if (!migrationTotal.value) return 0
return Math.round(100 * (Number(migrationCounts.value.enrolled) || 0) / migrationTotal.value)
})
const readyRows = computed(() => migrationRows.value.filter(row => row.state === 'ready'))
const allReadySelected = computed(() =>
readyRows.value.length > 0 && readyRows.value.every(row => selected.value.includes(row.assetid)))
function toggleAllReady(event) {
selected.value = event.target.checked ? readyRows.value.map(row => row.assetid) : []
}
function toggleMigrationState(state) {
migrationState.value = migrationState.value === state ? '' : state
reloadMigration()
}
async function loadMigration({ quiet = false } = {}) {
if (!quiet) migrationLoading.value = true
try {
const params = { page: migrationPage.value, perpage: PERPAGE }
if (migrationState.value) params.state = migrationState.value
if (migrationSearch.value) params.search = migrationSearch.value
const response = await api.get('/endpoints/enrollment/migration', { params })
const data = response.data.data || {}
if (data.target) target.value = data.target
migrationCounts.value = data.counts || {}
migrationRows.value = data.computers || []
waves.value = data.waves || []
migrationTotalPages.value = response.data.meta?.pagination?.totalpages || 1
// Drop selections that are no longer ready (queued by a wave, went offline).
const readyIds = new Set(readyRows.value.map(row => row.assetid))
selected.value = selected.value.filter(id => readyIds.has(id))
migrationError.value = ''
} catch (error) {
migrationError.value = apiError(error, 'Could not load migration status')
} finally {
migrationLoading.value = false
}
}
function reloadMigration() {
migrationPage.value = 1
loadMigration()
}
function goToMigrationPage(page) {
migrationPage.value = page
loadMigration()
}
let migrationSearchTimer = null
function debouncedMigrationSearch() {
clearTimeout(migrationSearchTimer)
migrationSearchTimer = setTimeout(reloadMigration, 300)
}
function waveConfirmText(count) {
return `Upgrade ${count} PC(s) to endpoint agent ${target.value}?\n\n`
+ 'Each PC downloads the new agent and restarts its service. Once the new '
+ 'agent is running it enrolls automatically with the shared key it already '
+ 'holds; no further action is needed unless it lands in Pending.'
}
async function startWave(body, count) {
if (!count) return
if (!confirm(waveConfirmText(count))) return
busy.value = true
try {
const response = await api.post('/endpoints/enrollment/migration/wave', body)
const data = response.data.data || {}
const queued = (data.queued || []).length
const skipped = data.skipped || []
if (queued) toast.success(`Wave ${data.waveid}: upgrade queued for ${queued} PC(s)`)
if (skipped.length) {
toast.info(`${skipped.length} skipped: `
+ skipped.slice(0, 5).map(item => `${item.assetid} (${item.reason})`).join(', '))
}
selected.value = []
await loadMigration({ quiet: true })
} catch (error) {
toast.error(apiError(error, 'Could not start the wave'))
} finally {
busy.value = false
}
}
function upgradeSelected() {
const assetids = [...selected.value]
return startWave({ assetids }, assetids.length)
}
function upgradeNext() {
const count = Math.min(waveSize, Number(migrationCounts.value.ready) || 0)
return startWave({ next: waveSize }, count)
}
function wavePercent(wave) {
if (!wave.size) return 0
return Math.round(100 * ((Number(wave.enrolled) || 0) + (Number(wave.failed) || 0)) / wave.size)
}
// ---- Credentials ------------------------------------------------------
const enrollmentSettings = ref({ sharedkeyallowed: true, autoenroll: true })
const enrollmentCounts = ref({})
const credentialRows = ref([])
const requests = ref([])
const enrollmentPage = ref(1)
const enrollmentTotalPages = ref(1)
const enrollmentState = ref('')
const enrollmentSearch = ref('')
const enrollmentLoading = ref(false)
const enrollmentError = ref('')
const approving = ref(null)
async function loadEnrollment({ quiet = false } = {}) {
if (!quiet) enrollmentLoading.value = true
try {
const params = { page: enrollmentPage.value, perpage: PERPAGE }
if (enrollmentState.value) params.state = enrollmentState.value
if (enrollmentSearch.value) params.search = enrollmentSearch.value
const response = await api.get('/endpoints/enrollment', { params })
const data = response.data.data || {}
enrollmentSettings.value = {
sharedkeyallowed: data.settings?.sharedkeyallowed !== false,
autoenroll: data.settings?.autoenroll !== false
}
enrollmentCounts.value = data.counts || {}
credentialRows.value = data.computers || []
requests.value = data.requests || []
enrollmentTotalPages.value = response.data.meta?.pagination?.totalpages || 1
enrollmentError.value = ''
} catch (error) {
enrollmentError.value = apiError(error, 'Could not load enrollment')
} finally {
enrollmentLoading.value = false
}
}
function reloadEnrollment() {
enrollmentPage.value = 1
loadEnrollment()
}
function goToEnrollmentPage(page) {
enrollmentPage.value = page
loadEnrollment()
}
let enrollmentSearchTimer = null
function debouncedEnrollmentSearch() {
clearTimeout(enrollmentSearchTimer)
enrollmentSearchTimer = setTimeout(reloadEnrollment, 300)
}
async function saveSettings(next) {
busy.value = true
try {
await api.put('/endpoints/enrollment/settings', next)
toast.success('Enrollment settings saved')
} catch (error) {
toast.error(apiError(error, 'Could not save enrollment settings'))
} finally {
busy.value = false
await loadEnrollment({ quiet: true })
}
}
function onSharedKeyToggle(event) {
const turningOn = event.target.checked
if (!turningOn) {
const notenrolled = Number(enrollmentCounts.value.notenrolled) || 0
const answer = prompt(
'Turn OFF the shared key?\n\n'
+ `${notenrolled} PC(s) are not enrolled yet. Every one of them stops `
+ 'reporting, stops receiving commands, and cannot enroll on its own until '
+ 'the shared key is turned back on or an admin approves its request.\n\n'
+ 'Type OFF to confirm.')
if (!answer || answer.trim().toUpperCase() !== 'OFF') {
event.target.checked = true
return
}
}
return saveSettings({
sharedkeyallowed: turningOn,
autoenroll: enrollmentSettings.value.autoenroll
})
}
function onAutoEnrollToggle(event) {
return saveSettings({
sharedkeyallowed: enrollmentSettings.value.sharedkeyallowed,
autoenroll: event.target.checked
})
}
function openApprove(request) {
const options = []
if (request.matchedassetid) {
options.push({ assetid: request.matchedassetid, label: `${request.hostname} (matched)` })
}
approving.value = {
requestid: request.requestid,
hostname: request.hostname,
assetid: request.matchedassetid || null,
search: '',
options
}
}
let assetSearchTimer = null
function debouncedAssetSearch() {
clearTimeout(assetSearchTimer)
assetSearchTimer = setTimeout(searchAssets, 300)
}
async function searchAssets() {
const current = approving.value
if (!current || !current.search) return
try {
const response = await computersApi.list({ search: current.search, perpage: 20 })
const found = (response.data.data || []).map(item => ({
assetid: item.assetid,
label: `${item.computer?.hostname || item.name || item.assetid}`
+ (item.serialnumber ? ` (${item.serialnumber})` : '')
}))
const keep = current.options.filter(option => option.assetid === current.assetid)
const merged = [...keep]
found.forEach(option => {
if (!merged.some(existing => existing.assetid === option.assetid)) merged.push(option)
})
current.options = merged
} catch (error) {
toast.error(apiError(error, 'Could not search computers'))
}
}
async function approve() {
const current = approving.value
if (!current || !current.assetid) return
const option = current.options.find(item => item.assetid === current.assetid)
if (!confirm(`Approve the request from ${current.hostname} and issue a credential for `
+ `${option ? option.label : 'asset ' + current.assetid}?`)) return
busy.value = true
try {
await api.post(`/endpoints/enrollment/requests/${current.requestid}/approve`,
{ assetid: current.assetid })
toast.success(`Approved ${current.hostname}`)
approving.value = null
await loadEnrollment({ quiet: true })
} catch (error) {
toast.error(apiError(error, 'Could not approve the request'))
} finally {
busy.value = false
}
}
async function reject(request) {
if (!confirm(`Reject the enrollment request from ${request.hostname}?`)) return
busy.value = true
try {
await api.post(`/endpoints/enrollment/requests/${request.requestid}/reject`)
toast.success(`Rejected ${request.hostname}`)
await loadEnrollment({ quiet: true })
} catch (error) {
toast.error(apiError(error, 'Could not reject the request'))
} finally {
busy.value = false
}
}
async function revoke(row) {
const fallback = enrollmentSettings.value.sharedkeyallowed
? 'It falls back to the shared key and enrolls again on its own.'
: 'The shared key is off, so it cannot report until an admin approves its new request.'
if (!confirm(`Un-enroll ${row.hostname}?\n\nIts credential stops working immediately. ${fallback}`)) return
busy.value = true
try {
await api.post(`/endpoints/enrollment/assets/${row.assetid}/revoke`)
toast.success(`Un-enrolled ${row.hostname}`)
await loadEnrollment({ quiet: true })
} catch (error) {
toast.error(apiError(error, 'Could not un-enroll'))
} finally {
busy.value = false
}
}
// ---- Shared -----------------------------------------------------------
function stateLabel(state) {
return {
enrolled: 'Enrolled',
notenrolled: 'Not enrolled',
pending: 'Pending',
revoked: 'Revoked',
upgrading: 'Upgrading',
ready: 'Ready',
blocked: 'Blocked',
notreporting: 'Not reporting'
}[state] || state || '-'
}
function formatDate(value) {
if (!value) return '-'
return new Date(value).toLocaleString()
}
// Waves progress on their own; poll while this tab is open.
let pollTimer = null
function startPolling() {
stopPolling()
pollTimer = setInterval(() => {
if (tab.value === 'migration' && !busy.value) loadMigration({ quiet: true })
}, POLL_MS)
}
function stopPolling() {
if (pollTimer) clearInterval(pollTimer)
pollTimer = null
}
watch(tab, value => {
if (value === 'migration') loadMigration()
else loadEnrollment()
})
onMounted(() => {
loadMigration()
loadEnrollment({ quiet: true })
startPolling()
})
onBeforeUnmount(() => {
stopPolling()
clearTimeout(migrationSearchTimer)
clearTimeout(enrollmentSearchTimer)
clearTimeout(assetSearchTimer)
})
</script>
<style scoped>
.enroll-tabs {
display: flex;
gap: 0.25rem;
border-bottom: 1px solid var(--border);
margin-bottom: 1rem;
}
.enroll-tab {
padding: 0.6rem 1.25rem;
background: none;
border: none;
border-bottom: 2px solid transparent;
font: inherit;
font-weight: 500;
color: var(--text-light);
cursor: pointer;
}
.enroll-tab:hover { color: var(--text); }
.enroll-tab.active {
color: var(--primary);
border-bottom-color: var(--primary);
}
.load-error {
padding: 0.75rem 0;
color: var(--danger);
}
.enroll-tiles {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(140px, 1fr));
gap: 0.75rem;
margin-bottom: 1rem;
}
.enroll-tile {
display: flex;
flex-direction: column;
align-items: flex-start;
background: var(--bg-card);
border: 1px solid var(--border);
border-radius: 0.25rem;
padding: 0.8rem 0.95rem;
font: inherit;
color: var(--text);
text-align: left;
cursor: pointer;
}
.enroll-tile.static { cursor: default; }
.enroll-tile:not(.static):hover,
.enroll-tile.on { border-color: var(--primary); }
.enroll-tile .n {
font-size: 1.6rem;
font-weight: 700;
line-height: 1.1;
font-variant-numeric: tabular-nums;
}
.enroll-tile .l {
font-size: 0.8rem;
color: var(--text-light);
}
.enroll-tile.ok .n { color: var(--success-dark); }
.enroll-tile.warn .n { color: var(--warning-dark); }
.enroll-tile.bad .n { color: var(--danger); }
[data-theme="dark"] .enroll-tile.ok .n { color: var(--success); }
[data-theme="dark"] .enroll-tile.warn .n { color: var(--warning); }
.progress-row {
display: flex;
align-items: center;
gap: 0.75rem;
margin-bottom: 1rem;
}
.progress-track {
flex: 1;
height: 10px;
background: var(--bg);
border: 1px solid var(--border);
border-radius: 5px;
overflow: hidden;
}
.progress-track.small {
height: 6px;
min-width: 80px;
}
.progress-fill {
height: 100%;
background: var(--success);
}
.progress-text {
font-size: 0.9rem;
color: var(--text-light);
white-space: nowrap;
}
.enroll-filter { max-width: 200px; }
.bulk-bar {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.5rem;
padding: 0.6rem 0.9rem;
margin-bottom: 0.75rem;
background: var(--bg-card);
border: 1px solid var(--border);
border-radius: 0.25rem;
}
.bulk-count { font-size: 0.9rem; margin-right: 0.25rem; }
.select-cell { width: 1%; }
.empty-cell {
text-align: center;
color: var(--text-light);
}
.state-chip {
display: inline-block;
font-size: 0.78rem;
font-weight: 500;
padding: 0.1rem 0.5rem;
border-radius: 10px;
background: var(--bg);
color: var(--text-light);
white-space: nowrap;
}
.state-chip.enrolled,
.state-chip.ready { color: var(--success-dark); }
.state-chip.pending,
.state-chip.upgrading { color: var(--warning-dark); }
.state-chip.blocked,
.state-chip.revoked { color: var(--danger); }
[data-theme="dark"] .state-chip.enrolled,
[data-theme="dark"] .state-chip.ready { color: var(--success); }
[data-theme="dark"] .state-chip.pending,
[data-theme="dark"] .state-chip.upgrading { color: var(--warning); }
.blockers {
font-size: 0.85rem;
color: var(--text-light);
}
.section-title { margin: 1.5rem 0 0.5rem; }
.checkbox-row {
display: flex;
align-items: flex-start;
gap: 0.5rem;
margin: 0.5rem 0;
}
.action-cell {
white-space: nowrap;
text-align: right;
}
.action-cell .btn + .btn { margin-left: 0.35rem; }
.approve-form {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 0.5rem;
}
.approve-form .form-control { max-width: 260px; }
.mono {
font-family: 'Courier New', monospace;
font-size: 13px;
}
</style>

33
plugins/endpoints/frontend/views/ServerCertSettings.vue
View File

@@ -66,6 +66,24 @@
</p>
</div>
<div class="form-group spki-pin">
<label>Public key pin (SPKI SHA-256)</label>
<p v-if="cert.spkisha256" class="field-hint mono">{{ cert.spkisha256 }}</p>
<p v-else class="field-hint">Not reported by this server.</p>
<button
v-if="cert.spkisha256"
class="btn btn-secondary btn-sm"
@click="copyPin"
>
{{ pinCopied ? 'Copied' : 'Copy pin' }}
</button>
<p class="field-hint">
Paste this into the GE-Enforce Set-ShopDBServerPins.ps1 entry
(-Pins) so agents 1.20 and later trust only this server's key.
It survives a renewal that keeps the same private key.
</p>
</div>
<div class="form-group">
<label>Issuer</label>
<p class="field-hint mono">{{ cert.issuer }}</p>
@@ -134,7 +152,11 @@
<div class="warning-banner" v-if="cert">
<strong>Warning:</strong> Generating a new certificate will replace the current one.
Agents using ServerCertThumbprint pinning will reject connections until you update
their appsettings.json with the new thumbprint.
their appsettings.json with the new thumbprint. A new certificate also has a new
key, so agents with ServerPins set will reject it until the pin entry carries the
new pin as well. Generating here gives no window to do that; to rotate without an
outage, compute the pin of a prepared certificate with Get-ShopDBServerPin.ps1, add
it to the pin entry alongside the old one, and upload that certificate below.
</div>
<div class="button-row">
@@ -278,6 +300,7 @@ const message = ref('')
const error = ref('')
const cert = ref(null)
const thumbprintCopied = ref(false)
const pinCopied = ref(false)
const generateForm = ref({
cn: 'localhost',
@@ -405,6 +428,14 @@ function copyThumbprint() {
}, 2000)
}
function copyPin() {
navigator.clipboard.writeText(cert.value.spkisha256)
pinCopied.value = true
setTimeout(() => {
pinCopied.value = false
}, 2000)
}
function formatDate(dateString) {
if (!dateString) return 'N/A'
const date = new Date(dateString)

200
plugins/endpoints/frontend/views/enrollmentSettings.spec.js Normal file
View File

@@ -0,0 +1,200 @@
/**
* Enrollment settings page (per-PC credentials).
*
* Pins the wire shapes the page sends, because the server contract is exact:
* a wave is {"next": n} or {"assetids": [...]}, an approval carries the chosen
* assetid, and turning the shared key off must tell the admin how many PCs are
* not enrolled yet before anything is sent.
*/
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { mount, flushPromises } from '@vue/test-utils'
vi.mock('@/api', () => ({
default: { get: vi.fn(), post: vi.fn(), put: vi.fn() },
computersApi: { list: vi.fn() }
}))
vi.mock('@/stores/auth', () => ({
useAuthStore: () => ({ isAdmin: true, hasPermission: () => true })
}))
vi.mock('@/composables/toast', () => ({
useToast: () => ({ success: vi.fn(), error: vi.fn(), info: vi.fn() })
}))
vi.mock('@/components/PaginationBar.vue', () => ({
default: { name: 'PaginationBar', template: '<div />' }
}))
import api, { computersApi } from '@/api'
import EnrollmentSettings from './EnrollmentSettings.vue'
function migrationPayload() {
return {
data: {
data: {
target: '1.20.0',
counts: { enrolled: 40, pending: 2, upgrading: 3, ready: 30, blocked: 5, notreporting: 20 },
computers: [
{ assetid: 11, hostname: 'PC-READY', agentversion: '1.19.2', lastreportedat: null, state: 'ready', blockers: [] },
{ assetid: 12, hostname: 'PC-BLOCKED', agentversion: '1.18.0', lastreportedat: null, state: 'blocked',
blockers: ['offline', 'no serial on record'] }
],
waves: [
{ waveid: 3, createdat: null, createdby: 'admin', size: 25, queued: 25, enrolled: 10, failed: 1, done: false }
]
},
meta: { pagination: { totalpages: 1 } }
}
}
}
function enrollmentPayload() {
return {
data: {
data: {
settings: { sharedkeyallowed: true, autoenroll: true },
counts: { enrolled: 40, notenrolled: 58, pending: 2 },
computers: [
{ assetid: 21, hostname: 'PC-ENROLLED', serialnumber: 'S1', state: 'enrolled',
credentialprefix: 'sdbe_abcdefg', issuedat: null, lastusedat: null, enrolledvia: 'auto' }
],
requests: [
{ requestid: 45, hostname: 'PC-NEW', serialnumber: 'S9', deviceid: null, reason: 'serial mismatch',
status: 'pending', matchedassetid: 77, requestedat: null, lastseenat: null, sourceip: null }
]
},
meta: { pagination: { totalpages: 1 } }
}
}
}
async function render() {
api.get.mockImplementation(path => Promise.resolve(
path === '/endpoints/enrollment/migration' ? migrationPayload() : enrollmentPayload()))
api.post.mockResolvedValue({ data: { data: { waveid: 4, queued: [11], skipped: [] } } })
api.put.mockResolvedValue({ data: { data: {} } })
const wrapper = mount(EnrollmentSettings)
await flushPromises()
return wrapper
}
async function openCredentials(wrapper) {
const tab = wrapper.findAll('.enroll-tab').find(button => button.text() === 'Credentials')
await tab.trigger('click')
await flushPromises()
}
describe('EnrollmentSettings', () => {
beforeEach(() => {
api.get.mockReset()
api.post.mockReset()
api.put.mockReset()
computersApi.list.mockReset()
})
afterEach(() => {
vi.unstubAllGlobals()
})
it('renders the migration counts and enrolled progress', async () => {
const wrapper = await render()
expect(wrapper.find('[data-count="enrolled"]').text()).toBe('40')
expect(wrapper.find('[data-count="ready"]').text()).toBe('30')
expect(wrapper.find('[data-count="blocked"]').text()).toBe('5')
expect(wrapper.find('[data-count="notreporting"]').text()).toBe('20')
expect(wrapper.find('.progress-text').text()).toContain('40 of 100 enrolled')
})
it('shows every blocker for a blocked PC', async () => {
const wrapper = await render()
const row = wrapper.findAll('.migration-row').find(r => r.text().includes('PC-BLOCKED'))
expect(row.find('.blockers').text()).toBe('offline, no serial on record')
})
it('only offers a checkbox on ready rows', async () => {
const wrapper = await render()
const rows = wrapper.findAll('.migration-row')
expect(rows[0].find('input[type="checkbox"]').exists()).toBe(true)
expect(rows[1].find('input[type="checkbox"]').exists()).toBe(false)
})
it('posts {"next":25} for Upgrade next 25 after a confirm naming the count', async () => {
const confirmSpy = vi.fn(() => true)
vi.stubGlobal('confirm', confirmSpy)
const wrapper = await render()
await wrapper.find('.upgrade-next').trigger('click')
await flushPromises()
expect(confirmSpy).toHaveBeenCalledTimes(1)
expect(confirmSpy.mock.calls[0][0]).toMatch(/Upgrade 25 PC\(s\) to endpoint agent 1\.20\.0/)
expect(confirmSpy.mock.calls[0][0]).toMatch(/enrolls automatically/)
expect(api.post).toHaveBeenCalledWith('/endpoints/enrollment/migration/wave', { next: 25 })
})
it('sends nothing when the wave confirm is declined', async () => {
vi.stubGlobal('confirm', vi.fn(() => false))
const wrapper = await render()
await wrapper.find('.upgrade-next').trigger('click')
await flushPromises()
expect(api.post).not.toHaveBeenCalled()
})
it('posts the selected assetids for Upgrade selected', async () => {
vi.stubGlobal('confirm', vi.fn(() => true))
const wrapper = await render()
await wrapper.findAll('.migration-row')[0].find('input[type="checkbox"]').setValue(true)
await wrapper.find('.upgrade-selected').trigger('click')
await flushPromises()
expect(api.post).toHaveBeenCalledWith('/endpoints/enrollment/migration/wave', { assetids: [11] })
})
it('approves a request with the matched asset by default', async () => {
vi.stubGlobal('confirm', vi.fn(() => true))
const wrapper = await render()
await openCredentials(wrapper)
await wrapper.find('.approve-open').trigger('click')
expect(wrapper.find('.asset-picker').element.value).toBe('77')
await wrapper.find('.approve-confirm').trigger('click')
await flushPromises()
expect(api.post).toHaveBeenCalledWith('/endpoints/enrollment/requests/45/approve', { assetid: 77 })
})
it('rejects a request after confirm', async () => {
vi.stubGlobal('confirm', vi.fn(() => true))
const wrapper = await render()
await openCredentials(wrapper)
await wrapper.find('.reject').trigger('click')
await flushPromises()
expect(api.post).toHaveBeenCalledWith('/endpoints/enrollment/requests/45/reject')
})
it('un-enrolls through the revoke route', async () => {
vi.stubGlobal('confirm', vi.fn(() => true))
const wrapper = await render()
await openCredentials(wrapper)
await wrapper.find('.revoke').trigger('click')
await flushPromises()
expect(api.post).toHaveBeenCalledWith('/endpoints/enrollment/assets/21/revoke')
})
it('shows the not-enrolled count before turning the shared key off', async () => {
const promptSpy = vi.fn(() => 'OFF')
vi.stubGlobal('prompt', promptSpy)
const wrapper = await render()
await openCredentials(wrapper)
await wrapper.find('.toggle-sharedkey').setValue(false)
await flushPromises()
expect(promptSpy).toHaveBeenCalledTimes(1)
expect(promptSpy.mock.calls[0][0]).toMatch(/58 PC\(s\) are not enrolled yet/)
expect(api.put).toHaveBeenCalledWith('/endpoints/enrollment/settings',
{ sharedkeyallowed: false, autoenroll: true })
})
it('leaves the shared key on when the confirmation is not typed', async () => {
vi.stubGlobal('prompt', vi.fn(() => 'yes'))
const wrapper = await render()
await openCredentials(wrapper)
const toggle = wrapper.find('.toggle-sharedkey')
await toggle.setValue(false)
await flushPromises()
expect(api.put).not.toHaveBeenCalled()
expect(toggle.element.checked).toBe(true)
})
})

287
plugins/printers/client/Report-PrintersToShopDB.ps1
View File

@@ -30,11 +30,19 @@
# lost its printers.
#
# AUTH: the collector API does NOT honor the GE-Enforce IP allowlist (that only
# covers the geenforce fetch/report endpoints). It needs a collector.ingest key,
# sent as the X-API-Key header. The key is read from HKLM:\SOFTWARE\GE\ShopDB
# CollectorKey (the same secret store Report-AssetToShopDB.ps1 uses; provisioned
# at imaging), or overridden via the manifest entry's Args -ApiKey. Never bake
# the key into the manifest JSON on the share.
# covers the geenforce fetch/report endpoints). Once the endpoint agent (1.20+)
# has enrolled this PC, its per-PC credential is read from the agent's
# SYSTEM-only credential.bin and sent as X-Endpoint-Credential; the server then
# refuses the shared key for this PC (401 SHARED_KEY_REFUSED). Before that, a
# collector.ingest key is sent as the X-API-Key header, read from
# HKLM:\SOFTWARE\GE\ShopDB CollectorKey (the same secret store
# Report-AssetToShopDB.ps1 uses; provisioned at imaging), or overridden via the
# manifest entry's Args -ApiKey. Never bake the key into the manifest JSON on
# the share.
#
# PINS: when HKLM:\SOFTWARE\GE\ShopDB ServerPins is set, the server's TLS key
# must match one of them or nothing is sent. See
# plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md.
#
# Runs every GE-Enforce cycle as a Type=PS1 / DetectionMethod=Always entry under
# the SYSTEM task. Always exits 0 so a printer problem never fails an
@@ -77,6 +85,227 @@ function Log([string]$msg) {
"$ts $msg" | Tee-Object -FilePath $logFile -Append | Out-Null
}
# ---------------------------------------------------------------------------
# Per-PC credential and server pin. Kept in step with the same block in
# Report-AssetToShopDB.ps1, Report-PrintersToShopDB.ps1 and
# upgrade-shopdbagent.ps1: each runs standalone, so the text is duplicated.
#
# Credential: the endpoint agent (1.20+) enrolls and stores its own credential
# in credential.bin, DPAPI LocalMachine, readable only by SYSTEM and
# Administrators. When present it is sent as X-Endpoint-Credential and the
# shared key is not sent. Without it the shared CollectorKey goes as X-API-Key.
#
# Pins: HKLM\SOFTWARE\GE\ShopDB ServerPins (REG_MULTI_SZ), lowercase hex
# SHA-256 of the server leaf certificate's SubjectPublicKeyInfo. When any pin
# is set the TLS handshake is accepted only if the leaf key matches one.
# ---------------------------------------------------------------------------
$ShopDbCredentialFile = Join-Path $env:ProgramData 'ShopDB\EndpointAgent\credential\credential.bin'
$ShopDbPinSource = @'
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
public static class ShopDbServerPinValidator
{
private static readonly HashSet<string> Pins = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
public static string LastPresented;
public static void Install(string[] pins)
{
Pins.Clear();
foreach (string pin in pins)
{
if (!String.IsNullOrEmpty(pin)) { Pins.Add(pin); }
}
ServicePointManager.ServerCertificateValidationCallback = Validate;
}
public static bool Validate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors errors)
{
if (Pins.Count == 0) { return errors == SslPolicyErrors.None; }
if (certificate == null) { return false; }
try
{
LastPresented = SpkiSha256Hex(certificate.GetRawCertData());
}
catch (Exception)
{
LastPresented = null;
return false;
}
return Pins.Contains(LastPresented);
}
// Walks the DER: Certificate > TBSCertificate > [0] version (optional),
// serialNumber, signature, issuer, validity, subject, subjectPublicKeyInfo.
public static string SpkiSha256Hex(byte[] der)
{
int length;
int position = EnterSequence(der, 0, out length);
position = EnterSequence(der, position, out length);
if (der[position] == 0xA0) { position = Skip(der, position); }
for (int field = 0; field < 5; field++) { position = Skip(der, position); }
if (der[position] != 0x30) { throw new FormatException("SubjectPublicKeyInfo not found"); }
int end = Skip(der, position);
using (SHA256 sha = SHA256.Create())
{
byte[] hash = sha.ComputeHash(der, position, end - position);
return BitConverter.ToString(hash).Replace("-", "").ToLowerInvariant();
}
}
private static int ReadLength(byte[] der, int position, out int length)
{
int first = der[position];
if (first < 0x80) { length = first; return position + 1; }
int count = first & 0x7F;
if (count == 0 || count > 4) { throw new FormatException("Unsupported DER length"); }
length = 0;
for (int i = 1; i <= count; i++) { length = (length << 8) | der[position + i]; }
return position + 1 + count;
}
private static int EnterSequence(byte[] der, int position, out int length)
{
if (der[position] != 0x30) { throw new FormatException("Expected SEQUENCE"); }
return ReadLength(der, position + 1, out length);
}
private static int Skip(byte[] der, int position)
{
int length;
int content = ReadLength(der, position + 1, out length);
return content + length;
}
}
'@
function Get-ShopDbServerPins {
# Both registry views, like CollectorKey. Anything not 64 hex chars is dropped.
$pins = @()
foreach ($view in @([Microsoft.Win32.RegistryView]::Registry64, [Microsoft.Win32.RegistryView]::Registry32)) {
try {
$base = [Microsoft.Win32.RegistryKey]::OpenBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine, $view)
try {
$key = $base.OpenSubKey('SOFTWARE\GE\ShopDB')
if ($key) {
try {
foreach ($value in @($key.GetValue('ServerPins'))) {
if ($null -eq $value) { continue }
$pin = ([string]$value -replace '[\s:]', '').ToLowerInvariant()
if ($pin -match '^[0-9a-f]{64}$' -and $pins -notcontains $pin) { $pins += $pin }
}
} finally { $key.Dispose() }
}
} finally { $base.Dispose() }
} catch {}
}
return ,$pins
}
function Enable-ShopDbServerPins([string[]]$Pins) {
# True when pinning is active. False means pins are set but could not be
# enforced; the caller must not send anything then.
if ($PSVersionTable.PSEdition -eq 'Core') {
# PowerShell 7 web cmdlets ignore ServicePointManager, so the check
# would silently not happen.
Log 'ERROR ServerPins is set; pin checking needs Windows PowerShell 5.1 (powershell.exe), not pwsh'
return $false
}
try {
if (-not ('ShopDbServerPinValidator' -as [type])) {
Add-Type -TypeDefinition $ShopDbPinSource -ErrorAction Stop
}
[ShopDbServerPinValidator]::Install($Pins)
return $true
} catch {
Log "ERROR ServerPins is set but pin checking could not be enabled: $($_.Exception.Message)"
return $false
}
}
function Get-ShopDbEndpointCredential([string]$TargetUrl) {
# Empty when there is none or it cannot be used for this URL.
if (-not (Test-Path -LiteralPath $ShopDbCredentialFile)) { return '' }
try {
Add-Type -AssemblyName System.Security -ErrorAction Stop
$protected = [System.IO.File]::ReadAllBytes($ShopDbCredentialFile)
$plain = [System.Security.Cryptography.ProtectedData]::Unprotect(
$protected, $null, [System.Security.Cryptography.DataProtectionScope]::LocalMachine)
$record = [System.Text.Encoding]::UTF8.GetString($plain) | ConvertFrom-Json
} catch {
Log "WARN could not read the endpoint credential ($ShopDbCredentialFile): $($_.Exception.Message)"
return ''
}
$credential = [string]$record.credential
if (-not $credential.StartsWith('sdbe_')) {
Log 'WARN endpoint credential file has no usable credential; ignoring it'
return ''
}
if (-not $TargetUrl.StartsWith('https://', [System.StringComparison]::OrdinalIgnoreCase)) {
Log "WARN not sending the endpoint credential over a non-https URL ($TargetUrl)"
return ''
}
# Never hand a credential to a server other than the one that issued it.
$issuer = ([string]$record.baseurl).Trim().TrimEnd('/')
if ($issuer -and -not ($TargetUrl.TrimEnd('/') + '/').StartsWith($issuer + '/', [System.StringComparison]::OrdinalIgnoreCase)) {
Log "WARN endpoint credential was issued by $issuer, not the target $TargetUrl; not sending it"
return ''
}
return $credential
}
function Get-ShopDbHttpFailure($ErrorRecord) {
# Status and error envelope code (data.error.code) from a failed request.
$status = 0
$code = ''
$text = ''
try { $status = [int]$ErrorRecord.Exception.Response.StatusCode } catch {}
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
$text = $ErrorRecord.ErrorDetails.Message
} else {
try {
$reader = New-Object System.IO.StreamReader($ErrorRecord.Exception.Response.GetResponseStream())
$text = $reader.ReadToEnd()
$reader.Dispose()
} catch {}
}
if ($text) {
try { $code = [string](($text | ConvertFrom-Json).data.error.code) } catch {}
}
$presented = ''
if ('ShopDbServerPinValidator' -as [type]) { $presented = [string][ShopDbServerPinValidator]::LastPresented }
return @{ Status = $status; Code = $code; PresentedPin = $presented }
}
function Write-ShopDbAuthFailure($Failure) {
switch ($Failure.Code) {
'CREDENTIAL_INVALID' {
Log ('ERROR 401 CREDENTIAL_INVALID: the server does not accept this PC''s endpoint credential ' +
'(revoked or unknown). The endpoint agent deletes it and re-enrolls on its own.')
}
'SHARED_KEY_REFUSED' {
Log ('ERROR 401 SHARED_KEY_REFUSED: the server refuses the shared key for this PC, because ' +
'the PC is enrolled or the shared key is switched off. Reports need the endpoint agent ' +
"credential: check agent 1.20+ is installed, running and enrolled ($ShopDbCredentialFile).")
}
'ASSET_MISMATCH' {
Log ('ERROR 403 ASSET_MISMATCH: this PC''s credential belongs to a different asset than the ' +
'hostname in the report. Check the hostname and the enrollment in ShopDB.')
}
default {
if ($Failure.Status -eq 0 -and $Failure.PresentedPin) {
Log "ERROR TLS refused: server key pin $($Failure.PresentedPin) is not in ServerPins"
}
}
}
}
$REGPATHS = @('HKLM:\SOFTWARE\GE\ShopDB', 'HKLM:\SOFTWARE\WOW6432Node\GE\ShopDB')
function Get-ShopdbRegValue([string]$name) {
@@ -107,9 +336,22 @@ if (-not $ApiUrl -and -not $WhatIfOnly) {
# collector key from the GE-Enforce secret store when not passed via Args.
if (-not $ApiKey) { $ApiKey = Get-ShopdbRegValue 'CollectorKey' }
if (-not $ApiKey -and -not $WhatIfOnly) {
Log 'ERROR no collector key (HKLM:\SOFTWARE\GE\ShopDB CollectorKey or -ApiKey). Skipping.'
exit 0
$EndpointCredential = ''
if (-not $WhatIfOnly) {
$EndpointCredential = Get-ShopDbEndpointCredential $ApiUrl
if (-not $EndpointCredential -and -not $ApiKey) {
Log 'ERROR no endpoint credential and no collector key (HKLM:\SOFTWARE\GE\ShopDB CollectorKey or -ApiKey). Skipping.'
exit 0
}
$serverPins = Get-ShopDbServerPins
if ($serverPins.Count -gt 0) {
if (-not (Enable-ShopDbServerPins $serverPins)) {
Log 'ERROR refusing to report without the server pin check. Skipping.'
exit 0
}
Log "server pin check on ($($serverPins.Count) pin(s))"
}
}
# Queues that are not devices: the Windows-supplied virtual printers plus the
@@ -301,14 +543,27 @@ if ($WhatIfOnly) {
}
Log ("POST {0} host={1} queues={2}" -f $ApiUrl, $Hostname, $reported.Count)
try {
$response = Invoke-RestMethod -Uri $ApiUrl -Method Post -Body $json `
-ContentType 'application/json' `
-Headers @{ 'X-API-Key' = $ApiKey } `
-TimeoutSec $TimeoutSec -ErrorAction Stop
Log ("RESPONSE {0}" -f ($response | ConvertTo-Json -Compress -Depth 4))
} catch {
Log "ERROR POST failed: $($_.Exception.Message)"
# Credential first; the shared key only when there is no credential, or when
# the server has just refused the credential (revoked: the shared key is
# admitted again for this PC until the agent re-enrolls). Never both at once.
$attempts = @()
if ($EndpointCredential) { $attempts += ,@('credential', @{ 'X-Endpoint-Credential' = $EndpointCredential }) }
if ($ApiKey) { $attempts += ,@('shared key', @{ 'X-API-Key' = $ApiKey }) }
foreach ($attempt in $attempts) {
try {
$response = Invoke-RestMethod -Uri $ApiUrl -Method Post -Body $json `
-ContentType 'application/json' `
-Headers $attempt[1] `
-TimeoutSec $TimeoutSec -ErrorAction Stop
Log ("RESPONSE ({0}) {1}" -f $attempt[0], ($response | ConvertTo-Json -Compress -Depth 4))
break
} catch {
Log "ERROR POST with $($attempt[0]) failed: $($_.Exception.Message)"
$failure = Get-ShopDbHttpFailure $_
Write-ShopDbAuthFailure $failure
if ($failure.Code -ne 'CREDENTIAL_INVALID') { break }
}
}
exit 0