Give each endpoint its own credential and fix the review findings on the server

Per-PC credentials (replaces the 09-14 client-certificate attempt):
- POST /api/endpoints/enroll issues a credential to a PC that matches exactly
  one computer by hostname and BIOS serial and has none yet. Anything else
  (unknown or duplicate hostname, serial mismatch, already enrolled) waits as
  a request an admin approves or rejects. The agent's persisted nonce binds an
  approved request to the PC that asked.
- One auth helper serves every agent route. With X-Endpoint-Credential the
  asset comes from the credential and a request naming another PC gets 403
  ASSET_MISMATCH. The shared key still works for PCs that have not enrolled,
  is refused (401 SHARED_KEY_REFUSED) for any PC that has, and can be switched
  off. A revoked credential gets 401 CREDENTIAL_INVALID so the agent
  re-enrolls.
- The core collector accepts the credential for the computers, printers and
  endpoints collectors through two new plugin hooks exposed on shopdb.api
  (contract 0.33.0), and refuses the shared key for enrolled PCs.
- Admin API: enrollment list, approve, reject, revoke (un-enroll), settings.
  Migration API: per-PC readiness with blockers, and upgrade waves that
  package the agent once and queue the existing agent upgrade per PC.
- Migration 0023 creates endpointcredentials, endpointenrollrequests and
  endpointmigrationwaves, drops the certificate columns from assets where
  test databases have them, and renames app_fingerprint to appfingerprint.
  0021 and 0022 keep their ids but no longer alter the core assets table.
- Removed: client_cert.py, certificate renewal, the unauthenticated
  /test/client-cert route, nginx client certificate forwarding and the agent
  certificate script. The client-cert header could be forged on IIS, the key
  was readable by users, and the shared key remained a fallback.
- GET /settings/server-cert returns the SPKI SHA-256 pin for GE-Enforce.

Review fixes:
- report_command_result used g without importing it; every result was a 500.
- collect_log refuses setup answer files, appsettings, agent directories and
  key files at enqueue.
- Fleet alerts: alerttimeout and alertfleet agree with the migration, the
  scope is validated and a malformed scope no longer reaches the whole fleet,
  and the ack race no longer raises NameError.
- A refused break-glass override leaves no hold and no audit row.
- Agent upgrade checks locks and permissions before repackaging, and writes
  the package atomically.
- Server certificate upload validates the pair before replacing files;
  generation no longer builds an openssl config from user input; audit rows
  are actually written.
- Health evaluation runs in a savepoint so a caught error cannot lose the
  check-in.
- Required-app policies are found by appid, survive renames, and deleting an
  app deactivates policies that still name it. The UI and API now say such a
  policy only takes effect inside a business policy.
- Backfill handles duplicate fingerprints and rolls back a failed batch.
- Plugin settings routes skip a plugin whose hook raises.
- The setup token is no longer printed to the log.
- The naming gate checks top-level files again and handles odd file names.
- Removed root scratch scripts, one of which carried a token.
- Rewrote the fleet alert scheduling and app delta tests on real fixtures.
This commit is contained in:
cproudlock
2026-09-15 12:03:53 -04:00
parent f484be8a33
commit 7480e4fe3a
58 changed files with 4485 additions and 2032 deletions

View File

@@ -15,11 +15,6 @@ http {
ssl_certificate /etc/nginx/certs/server.crt;
ssl_certificate_key /etc/nginx/certs/server.key;
# Client certificate validation (optional during rollout)
# optional_no_ca: accept cert but don't validate against CA
# Flask validates fingerprint against database
ssl_verify_client optional_no_ca;
# Increase timeouts for long-polling agents
proxy_connect_timeout 90s;
proxy_send_timeout 90s;
@@ -36,19 +31,8 @@ http {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
# Pass client certificate fingerprint to Flask
# SECURITY NOTE: $ssl_client_fingerprint outputs SHA1 (40 hex chars),
# not SHA256. This is a known nginx limitation. SHA1 is acceptable for
# certificate fingerprinting (not signing) in air-gapped environments
# with short-lived certs and revocation support.
# Format: "AB:CD:EF:..." (colon-separated hex, normalized in Flask)
# Future SHA256: requires custom nginx module or migrate to IIS
proxy_set_header X-Client-Cert-Fingerprint $ssl_client_fingerprint;
proxy_set_header X-Client-Cert-S-DN $ssl_client_s_dn;
}
# Access log includes client cert status
access_log /dev/stdout combined;
error_log /dev/stderr warn;
}

View File

@@ -118,17 +118,19 @@
was already required has an existing policy, so re-saving it is
not a new deployment and does not need re-acknowledging. -->
<div v-if="showsPolicyWarning" class="policy-warning">
<strong>This will install software on your fleet.</strong>
<strong>This creates a software install policy.</strong>
<p>
Marking <em>{{ form.appname || 'this application' }}</em> as required
on all PCs creates a configuration policy that installs it on every
non-server endpoint it applies to. Installation happens
automatically, without further prompting, on each machine as it
checks in.
non-server endpoint it applies to. The policy is not added to a
business policy automatically: it installs nothing until an admin
adds it to an active business policy. From then on, installation
happens without further prompting on each machine as it checks in.
</p>
<label>
<input type="checkbox" v-model="acknowledgePolicy" />
I understand this will install the application on all matched PCs.
I understand this creates a policy that will install the application
on all matched PCs once it is added to a business policy.
</label>
</div>
</div>

View File

@@ -6,7 +6,7 @@ import logging
from pathlib import Path
from typing import List, Dict, Optional, Type
from flask import Flask, Blueprint
from flask import Flask, Blueprint, g
import click
from shopdb.plugins.base import BasePlugin, PluginMeta
@@ -454,6 +454,7 @@ class ComputersPlugin(BasePlugin):
"""
return {
'identityfield': 'hostname',
'acceptsagentcredential': True,
'fields': {
'hostname': {'type': 'string', 'required': True},
'machinenumber': {'type': 'string'},
@@ -724,6 +725,13 @@ class ComputersPlugin(BasePlugin):
comp = (Computer.query.join(Asset, Asset.assetid == Computer.assetid)
.filter(Asset.assetnumber.ilike(hostname)).first())
# A per-PC credential reports for its own PC only, and never creates one.
if (getattr(g, 'agentauth', None) == 'credential'
and (comp is None
or comp.assetid != getattr(g, 'agentassetid', None))):
raise PermissionError(
'The payload names a different endpoint than the credential')
action = 'updated'
if not comp:
atype = AssetType.query.filter_by(assettype='computer').first()

View File

@@ -15,6 +15,14 @@ kept. Deleting someone's customised enforcement rule because a checkbox changed
on a different screen would be a genuinely destructive surprise, and the box is
not a reliable signal that they wanted that.
Not attached to a Business Policy
---------------------------------
The generated policy is NOT added to any Business Policy. The resolver only
applies Configuration Policies reached through an active Business Policy, so
until an admin adds it to one, the generated policy installs nothing. The API
response and the acknowledgement text say so rather than promising a fleet
install that will not happen.
Scope
-----
Generated policies use the `allpcs` node, not `true`: "required on all PCs"
@@ -48,13 +56,58 @@ def auto_policy_name(application):
return '{} {}'.format(AUTO_POLICY_PREFIX, application.appname)
def _action_appid(action_row):
"""The appid an action's params name, or None."""
params = action_row.params
if not isinstance(params, dict):
return None
try:
return int(params.get('appid'))
except (TypeError, ValueError):
return None
def policies_referencing_app(appid):
"""Every ConfigurationPolicy with an action whose params name this appid.
Params are a JSON text column (MySQL 5.6 has no JSON type to query), so the
match is done in Python over the actions that carry params at all.
"""
if appid is None:
return []
found = {}
rows = (ConfigurationPolicyAction.query
.filter(ConfigurationPolicyAction.params.isnot(None))
.all())
for action_row in rows:
if _action_appid(action_row) == int(appid):
policy = action_row.configurationpolicy
if policy is not None:
found[policy.configurationpolicyid] = policy
return [found[key] for key in sorted(found)]
def find_auto_policy(application):
"""Return the policy generated for this app, whoever owns it now.
Matches on name, since that is the only durable link - there is no FK from
a policy back to an application. Returns rows that have since been flipped
to 'local' too, so callers can tell "customised" from "absent".
Keyed on the appid in the generated update_app action's params, not on the
name: the name embeds appname, so renaming the application used to orphan
its policy (the next untick could not find it, and a re-tick generated a
second one). A policy counts as generated when it still carries the autoapp
source or the generated name prefix, so a row flipped to 'local' by an edit
is still found and callers can tell "customised" from "absent". The name
lookup stays as a fallback for a row whose params were edited away.
"""
candidates = [
policy for policy in policies_referencing_app(application.appid)
if policy.source == POLICY_SOURCE_AUTOAPP
or (policy.name or '').startswith(AUTO_POLICY_PREFIX)
]
if candidates:
candidates.sort(key=lambda policy: (
policy.source != POLICY_SOURCE_AUTOAPP,
policy.configurationpolicyid))
return candidates[0]
return ConfigurationPolicy.query.filter_by(
name=auto_policy_name(application)
).first()
@@ -75,7 +128,13 @@ def sync_required_app_policy(application):
if existing is not None:
# Already enforced. Deliberately does NOT rewrite a customised row
# back to our defaults; re-ticking a box should not silently revert
# an admin's edits.
# an admin's edits. A still-generated row does follow an app rename
# so its name keeps describing what it installs.
wanted = auto_policy_name(application)
if (existing.source == POLICY_SOURCE_AUTOAPP
and existing.name != wanted
and ConfigurationPolicy.query.filter_by(name=wanted).first() is None):
existing.name = wanted
return 'unchanged'
policy = ConfigurationPolicy(
@@ -122,3 +181,22 @@ def disown_if_generated(policy):
policy.source = POLICY_SOURCE_LOCAL
return True
return False
def deactivate_policies_for_app(appid):
"""Deactivate every remaining policy whose actions name a deleted appid.
Called when an application is deleted, after the generated policy has been
withdrawn. What is left is either the customised ('kept') generated policy
or a hand-authored one; both would keep trying to install an application
that no longer exists. They are deactivated rather than deleted - they are
someone's work - and returned so the API can name them.
"""
deactivated = []
for policy in policies_referencing_app(appid):
if policy in db.session.deleted:
continue
if policy.isactive:
policy.isactive = False
deactivated.append(policy.name)
return deactivated

View File

@@ -9,5 +9,7 @@ from . import siteagent # noqa: F401
from . import agentupload # noqa: F401
# Attaches the monitor catalog CRUD and resolve routes to endpoints_bp.
from . import monitors # noqa: F401
# Attaches per-PC credential enrollment, admin and migration routes.
from . import enrollment # noqa: F401
__all__ = ['endpoints_bp']

View File

@@ -0,0 +1,196 @@
"""Agent authentication: per-PC credentials, with the shared key as fallback.
Every agent-facing endpoints route authenticates through authenticate_agent()
and, when the route names a PC, binds the caller through bind_hostname().
X-Endpoint-Credential The PC's own secret. The server derives the asset from
it; a hostname or assetid in the request that names a
different asset is refused with ASSET_MISMATCH.
X-API-Key The shared fleet key (or a collector-scoped managed
token). Accepted only while
plugin.endpoints.sharedkeyallowed is on, and never for a
PC that holds an active credential.
Only the sha256 of a credential is stored. The full secret exists once, in the
enroll response.
"""
import hashlib
import hmac
import re
import secrets
from datetime import datetime, timedelta, timezone
from flask import g, request
from shopdb.api import (
db, Setting, error_response, ErrorCodes, collector_authorized,
)
CREDENTIAL_HEADER = 'X-Endpoint-Credential'
CREDENTIAL_PREFIX = 'sdbe_'
# sdbe_ + 43 chars base64url (32 random bytes, unpadded).
_CREDENTIAL_PATTERN = re.compile(r'^sdbe_[A-Za-z0-9_-]{43}$')
DISPLAY_PREFIX_LENGTH = 12
SHARED_KEY_SETTING = 'plugin.endpoints.sharedkeyallowed'
AUTO_ENROLL_SETTING = 'plugin.endpoints.autoenroll'
CODE_CREDENTIAL_INVALID = 'CREDENTIAL_INVALID'
CODE_SHARED_KEY_REFUSED = 'SHARED_KEY_REFUSED'
CODE_ASSET_MISMATCH = 'ASSET_MISMATCH'
AUTH_CREDENTIAL = 'credential'
AUTH_SHARED_KEY = 'sharedkey'
# lastusedat is a display value, not an audit trail: one write per 5 minutes
# per PC keeps the long-poll from turning every claim into an UPDATE.
_TOUCH_INTERVAL = timedelta(minutes=5)
def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def generate_credential():
return CREDENTIAL_PREFIX + secrets.token_urlsafe(32)
def hash_credential(secret):
return hashlib.sha256(secret.encode('utf-8')).hexdigest()
def shared_key_allowed():
return Setting.get(SHARED_KEY_SETTING, default=True) is not False
def auto_enroll_enabled():
return Setting.get(AUTO_ENROLL_SETTING, default=True) is not False
def active_credential_for(assetid):
from ..models import EndpointCredential
if assetid is None:
return None
return (EndpointCredential.query
.filter(EndpointCredential.assetid == assetid,
EndpointCredential.revokedat.is_(None))
.first())
def resolve_credential(secret, touch=True):
"""Asset id for an active credential, else None."""
from ..models import EndpointCredential
if not secret or not _CREDENTIAL_PATTERN.match(secret):
return None
digest = hash_credential(secret)
row = EndpointCredential.query.filter_by(credentialhash=digest).first()
if row is None or row.revokedat is not None:
return None
if not hmac.compare_digest(row.credentialhash, digest):
return None
if touch:
now = _naive_utcnow()
if row.lastusedat is None or now - row.lastusedat >= _TOUCH_INTERVAL:
row.lastusedat = now
db.session.commit()
return row.assetid
def resolve_hostname(hostname):
"""Asset id of the one computer with this hostname (case-insensitive)."""
from plugins.computers.models import Computer
hostname = (hostname or '').strip()
if not hostname:
return None
computer = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.first())
return computer.assetid if computer is not None else None
def shared_key_refused_for(hostname):
"""True when the shared key may not report for this hostname."""
if not shared_key_allowed():
return True
assetid = resolve_hostname(hostname)
return assetid is not None and active_credential_for(assetid) is not None
def credential_invalid_response():
return error_response(
CODE_CREDENTIAL_INVALID,
'Endpoint credential is unknown or revoked; enroll again',
http_code=401)
def shared_key_refused_response():
return error_response(
CODE_SHARED_KEY_REFUSED,
'The shared key is not accepted for this endpoint; use its credential',
http_code=401)
def asset_mismatch_response():
return error_response(
CODE_ASSET_MISMATCH,
'The request names a different endpoint than the credential',
http_code=403)
def authenticate_agent():
"""Authorize an agent request. Returns None when authorized, else a response.
Sets g.agentauth ('credential' or 'sharedkey') and g.agentassetid (the
credential's asset, or None until bind_hostname resolves one).
"""
g.agentauth = None
g.agentassetid = None
secret = request.headers.get(CREDENTIAL_HEADER)
if secret:
assetid = resolve_credential(secret.strip())
if assetid is None:
return credential_invalid_response()
g.agentauth = AUTH_CREDENTIAL
g.agentassetid = assetid
return None
error = collector_authorized('endpoints')
if error is not None:
return error
if not shared_key_allowed():
return shared_key_refused_response()
g.agentauth = AUTH_SHARED_KEY
return None
def is_credential_caller():
return getattr(g, 'agentauth', None) == AUTH_CREDENTIAL
def bind_hostname(hostname):
"""The asset this request acts for. Returns (assetid, error_response).
Credential: the credential's asset; a hostname naming another asset is an
ASSET_MISMATCH, and a missing hostname is fine.
Shared key: the hostname's asset (None when absent or unknown, for the
route to answer); an enrolled PC is refused.
"""
hostname = (hostname or '').strip()
if is_credential_caller():
if hostname and resolve_hostname(hostname) != g.agentassetid:
return None, asset_mismatch_response()
return g.agentassetid, None
if not hostname:
return None, None
assetid = resolve_hostname(hostname)
if assetid is not None and active_credential_for(assetid) is not None:
return None, shared_key_refused_response()
g.agentassetid = assetid
return assetid, None

View File

@@ -14,6 +14,7 @@ hand-maintained; an operator never edits these rows.
"""
import os
import uuid
from shopdb.api import Application, application_package_location, db
@@ -91,11 +92,19 @@ def register_current_agent_package():
os.makedirs(os.path.dirname(path), exist_ok=True)
# Write to a temp name and replace, so an interrupted write cannot leave a
# truncated archive at the path the row already advertises. os.replace is
# atomic within a filesystem.
temppath = path + '.tmp'
with open(temppath, 'wb') as handle:
handle.write(payload)
os.replace(temppath, path)
# atomic within a filesystem. The temp name is unique per call so two
# concurrent pushes (a batch wave) cannot interleave writes into one file.
temppath = '{}.{}.tmp'.format(path, uuid.uuid4().hex)
try:
with open(temppath, 'wb') as handle:
handle.write(payload)
os.replace(temppath, path)
except BaseException:
try:
os.remove(temppath)
except OSError:
pass
raise
_directory, urlprefix = application_package_location()
app.installpath = '{}{}/{}'.format(

View File

@@ -23,6 +23,7 @@ substitution done after hashing would fail the agent's verification.
import hashlib
import os
import uuid
from shopdb.api import Script, db, script_body_location
@@ -115,10 +116,17 @@ def seed_upgrade_script():
os.makedirs(_script_dir(), exist_ok=True)
# Temp-then-replace so an interrupted write cannot leave a truncated
# body at the path the row's digest already claims to describe.
temppath = path + '.tmp'
with open(temppath, 'wb') as handle:
handle.write(body)
os.replace(temppath, path)
temppath = '{}.{}.tmp'.format(path, uuid.uuid4().hex)
try:
with open(temppath, 'wb') as handle:
handle.write(body)
os.replace(temppath, path)
except BaseException:
try:
os.remove(temppath)
except OSError:
pass
raise
script.filename = filename
script.sha256 = digest

View File

@@ -1,103 +0,0 @@
"""Server-side certificate generation for agent self-renewal.
Agents check certificate expiry on startup and request renewal when < 30 days
remaining. Server generates new self-signed certificate, computes fingerprint,
updates database, and returns certificate as password-protected PFX to agent.
"""
import hashlib
import secrets
from datetime import datetime, timedelta, timezone
from flask import current_app
from cryptography import x509
from cryptography.x509.oid import NameOID, ExtendedKeyUsageOID
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.backends import default_backend
def generate_client_certificate(hostname, validity_days=730):
"""Generate self-signed client certificate for endpoint agent.
Args:
hostname: Computer hostname (used as CN)
validity_days: Certificate validity period (default 730 = 2 years)
Returns:
tuple: (pfx_bytes, pfx_password, sha1_fingerprint, expires_at)
- pfx_bytes: Password-protected PFX/P12 certificate with private key
- pfx_password: Random password for PFX (agent discards after import)
- sha1_fingerprint: 40-character lowercase hex SHA1 fingerprint
- expires_at: Certificate expiration datetime (UTC)
"""
# Generate RSA private key (2048-bit for compatibility with Windows)
private_key = rsa.generate_private_key(
public_exponent=65537,
key_size=2048,
backend=default_backend()
)
# Certificate subject and issuer (self-signed, so both are the same)
subject = issuer = x509.Name([
x509.NameAttribute(NameOID.COMMON_NAME, hostname),
x509.NameAttribute(NameOID.ORGANIZATION_NAME, 'ShopDB Endpoint Fleet'),
x509.NameAttribute(NameOID.ORGANIZATIONAL_UNIT_NAME, 'Endpoint Agents'),
])
# Certificate validity period
now = datetime.now(timezone.utc)
not_valid_before = now
not_valid_after = now + timedelta(days=validity_days)
# Build certificate
cert = x509.CertificateBuilder().subject_name(
subject
).issuer_name(
issuer
).public_key(
private_key.public_key()
).serial_number(
x509.random_serial_number()
).not_valid_before(
not_valid_before
).not_valid_after(
not_valid_after
).add_extension(
# Client authentication extended key usage
x509.ExtendedKeyUsage([ExtendedKeyUsageOID.CLIENT_AUTH]),
critical=True,
).add_extension(
# Subject Alternative Name (hostname as DNS name)
x509.SubjectAlternativeName([x509.DNSName(hostname)]),
critical=False,
).add_extension(
# Basic constraints: not a CA
x509.BasicConstraints(ca=False, path_length=None),
critical=True,
).sign(private_key, hashes.SHA256(), default_backend())
# Compute SHA1 fingerprint (matches nginx $ssl_client_fingerprint)
cert_der = cert.public_bytes(serialization.Encoding.DER)
sha1_hash = hashlib.sha1(cert_der).hexdigest()
# Generate random PFX password (agent uses once during import, then discards)
pfx_password = secrets.token_urlsafe(32)
# Serialize to PFX/P12 format with password protection
# Windows Certificate Store imports PFX format
pfx_bytes = serialization.pkcs12.serialize_key_and_certificates(
name=hostname.encode('utf-8'),
key=private_key,
cert=cert,
cas=None, # No CA chain for self-signed
encryption_algorithm=serialization.BestAvailableEncryption(
pfx_password.encode('utf-8')
)
)
current_app.logger.info(
f'[cert_renewal] Generated certificate for {hostname}: '
f'fingerprint={sha1_hash[:16]}..., expires={not_valid_after}')
return (pfx_bytes, pfx_password, sha1_hash, not_valid_after.replace(tzinfo=None))

View File

@@ -0,0 +1,789 @@
"""Per-PC endpoint credentials: enrollment, administration and fleet migration.
Agent-facing:
POST /enroll issue a credential or record a pending request
POST /collect the generic collector ingest, under this prefix
Admin (endpoints.fleet to read, admin role to change):
GET /enrollment credential state per computer + requests
POST /enrollment/requests/<id>/approve | reject
POST /enrollment/assets/<assetid>/revoke
PUT /enrollment/settings
GET /enrollment/migration readiness of every computer for agent 1.20
POST /enrollment/migration/wave push the agent upgrade to a batch of PCs
A PC moves onto its own credential by running an agent that enrolls. The
migration surface computes, per computer, whether the existing agent upgrade
can be pushed to it, and pushes it in waves through the same enqueue path the
single-asset agentupgrade route uses.
"""
import hmac
import re
from collections import Counter
from datetime import datetime, timedelta, timezone
from flask import request
from flask_jwt_extended import current_user
from shopdb.api import (
db, Asset, AuditLog, Script, error_response, success_response, ErrorCodes,
require_permission, require_role, collector_authorized, collector_ingest,
client_ip, get_pagination_params,
)
from ..commands import (
ACTIVE_LOCK_FOR, ALLOWED_ACTIONS_FOR, agent_meets_minimum,
minimum_agent_version_for, parse_agent_version,
)
from ..models import (
EndpointCommand, EndpointCredential, EndpointEnrollRequest,
EndpointMigrationWave, EndpointState,
)
from . import agentauth
from .routes import endpoints_bp
# First agent that enrolls itself.
CREDENTIAL_AGENT_VERSION = '1.20.0'
CODE_ENROLL_NONCE_MISMATCH = 'ENROLL_NONCE_MISMATCH'
_NONCE_PATTERN = re.compile(r'^[A-Za-z0-9_-]{32,128}$')
REASON_UNKNOWN_HOSTNAME = 'unknown hostname'
REASON_DUPLICATE_HOSTNAME = 'duplicate hostname'
REASON_NO_SERIAL = 'no serial on record'
REASON_SERIAL_MISMATCH = 'serial mismatch'
REASON_ALREADY_ENROLLED = 'already enrolled'
REASON_AWAITING_APPROVAL = 'awaiting approval'
REASON_NONCE_CHANGED = 'nonce changed'
REASON_REJECTED = 'rejected'
MIGRATION_STATES = ('enrolled', 'pending', 'upgrading', 'ready', 'blocked',
'notreporting')
ENROLLMENT_STATES = ('enrolled', 'notenrolled', 'pending', 'revoked')
BLOCKER_OFFLINE = 'offline'
BLOCKER_NO_SERIAL = 'no serial on record'
BLOCKER_DUPLICATE_HOSTNAME = 'duplicate hostname'
BLOCKER_LOCKED = 'commands locked'
BLOCKER_NO_PACKAGE = 'no agent package in store'
BLOCKER_TOO_OLD = 'agent too old for remote upgrade'
BLOCKER_POLICY = 'run_script not permitted by policy'
BLOCKER_AWAITING_ENROLLMENT = 'awaiting enrollment'
_OFFLINE_AFTER = timedelta(hours=24)
_WAVE_MAXIMUM = 100
_RECENT_REQUESTS = 50
def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def _utciso(value):
return value.isoformat() + 'Z' if value else None
def _username():
return getattr(current_user, 'username', None)
def _computer_model():
from plugins.computers.models import Computer
return Computer
# ---------------------------------------------------------------------------
# Agent-facing
# ---------------------------------------------------------------------------
@endpoints_bp.route('/collect', methods=['POST'])
def agent_collect():
"""Agent telemetry ingest. Same handler as POST /api/collector/endpoints."""
return collector_ingest('endpoints')
def _pending(row, reason, matchedassetid=None):
row.status = EndpointEnrollRequest.STATUS_PENDING
row.reason = reason
row.matchedassetid = matchedassetid
db.session.commit()
return success_response(
{'status': 'pending', 'requestid': row.requestid, 'reason': reason},
http_code=202)
def issue_credential(assetid, enrolledvia):
"""Issue (or re-issue after revoke) the credential for one asset.
Returns the secret. Only its hash is stored. Caller commits.
"""
secret = agentauth.generate_credential()
row = EndpointCredential.query.filter_by(assetid=assetid).first()
if row is None:
row = EndpointCredential(assetid=assetid)
db.session.add(row)
row.credentialhash = agentauth.hash_credential(secret)
row.credentialprefix = secret[:agentauth.DISPLAY_PREFIX_LENGTH]
row.enrolledvia = enrolledvia
row.issuedat = _naive_utcnow()
row.lastusedat = None
row.revokedat = None
row.revokedby = None
return secret, row
def _issue(row, assetid, enrolledvia):
secret, credential = issue_credential(assetid, enrolledvia)
row.status = EndpointEnrollRequest.STATUS_ISSUED
row.matchedassetid = assetid
row.reason = None
AuditLog.log('created', 'EndpointCredential', entityid=assetid,
entityname=row.hostname,
details='endpoint credential {} issued ({}), request {}'.format(
credential.credentialprefix, enrolledvia, row.requestid))
db.session.commit()
return success_response(
{'status': 'enrolled', 'credential': secret, 'assetid': assetid},
message='Endpoint enrolled', http_code=201)
def _text_field(payload, name, maximum, required):
value = payload.get(name)
if value is None:
value = ''
if not isinstance(value, str):
return None, f'{name} must be a string'
value = value.strip()
if required and not value:
return None, f'{name} is required'
if len(value) > maximum:
return None, f'{name} must be at most {maximum} characters'
return value, None
@endpoints_bp.route('/enroll', methods=['POST'])
def enroll_endpoint():
"""A PC asks for its own credential.
Shared-key authenticated while plugin.endpoints.sharedkeyallowed is on.
With it off the route takes no auth and only hands out a credential an
admin already approved for this exact hostname, serial and nonce.
"""
sharedkeyauthenticated = False
if agentauth.shared_key_allowed():
error = collector_authorized('endpoints')
if error is not None:
return error
sharedkeyauthenticated = True
payload = request.get_json(silent=True)
if not isinstance(payload, dict):
return error_response(ErrorCodes.VALIDATION_ERROR, 'No data provided')
hostname, problem = _text_field(payload, 'hostname', 255, True)
if problem is None:
serialnumber, problem = _text_field(payload, 'serialnumber', 128, True)
if problem is None:
deviceid, problem = _text_field(payload, 'deviceid', 64, False)
if problem is None:
agentversion, problem = _text_field(payload, 'agentversion', 32, False)
if problem is None:
nonce = payload.get('enrollnonce')
if not isinstance(nonce, str) or not _NONCE_PATTERN.match(nonce):
problem = 'enrollnonce must be 32-128 characters of A-Z a-z 0-9 _ -'
if problem is not None:
return error_response(ErrorCodes.VALIDATION_ERROR, problem)
now = _naive_utcnow()
sourceip = client_ip(request)
row = (EndpointEnrollRequest.query
.filter(db.func.lower(EndpointEnrollRequest.hostname) == hostname.lower(),
db.func.lower(EndpointEnrollRequest.serialnumber)
== serialnumber.lower())
.order_by(EndpointEnrollRequest.requestid.desc())
.first())
if row is None:
row = EndpointEnrollRequest(
hostname=hostname, serialnumber=serialnumber, enrollnonce=nonce,
status=EndpointEnrollRequest.STATUS_PENDING, requestedat=now)
db.session.add(row)
else:
samenonce = hmac.compare_digest(row.enrollnonce, nonce)
if row.status == EndpointEnrollRequest.STATUS_APPROVED:
row.lastseenat = now
row.sourceip = sourceip
if not samenonce:
db.session.commit()
return error_response(
CODE_ENROLL_NONCE_MISMATCH,
'This enrollment was approved for a different agent '
'installation', http_code=403)
if agentauth.active_credential_for(row.matchedassetid) is not None:
return _pending(row, REASON_ALREADY_ENROLLED, row.matchedassetid)
if row.matchedassetid is None or db.session.get(Asset, row.matchedassetid) is None:
return _pending(row, REASON_UNKNOWN_HOSTNAME)
return _issue(row, row.matchedassetid,
EndpointCredential.ENROLLED_APPROVED)
if row.status == EndpointEnrollRequest.STATUS_PENDING and not samenonce:
# First nonce wins; a second installation needs an admin.
row.lastseenat = now
row.sourceip = sourceip
return _pending(row, REASON_NONCE_CHANGED, row.matchedassetid)
if row.status == EndpointEnrollRequest.STATUS_REJECTED and samenonce:
row.lastseenat = now
row.sourceip = sourceip
db.session.commit()
return success_response(
{'status': 'pending', 'requestid': row.requestid,
'reason': REASON_REJECTED}, http_code=202)
if row.status != EndpointEnrollRequest.STATUS_PENDING:
# Issued earlier (credential since revoked) or rejected for another
# installation: a new enrollment cycle bound to this nonce.
row.enrollnonce = nonce
row.requestedat = now
row.decidedat = None
row.decidedby = None
row.lastseenat = now
row.sourceip = sourceip
if deviceid:
row.deviceid = deviceid
if agentversion:
row.agentversion = agentversion
db.session.flush()
Computer = _computer_model()
matches = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.all())
if not matches:
return _pending(row, REASON_UNKNOWN_HOSTNAME)
if len(matches) > 1:
return _pending(row, REASON_DUPLICATE_HOSTNAME)
assetid = matches[0].assetid
asset = db.session.get(Asset, assetid)
recorded = ((asset.serialnumber if asset is not None else None) or '').strip()
if not recorded:
return _pending(row, REASON_NO_SERIAL, assetid)
if recorded.lower() != serialnumber.lower():
return _pending(row, REASON_SERIAL_MISMATCH, assetid)
if agentauth.active_credential_for(assetid) is not None:
return _pending(row, REASON_ALREADY_ENROLLED, assetid)
if sharedkeyauthenticated and agentauth.auto_enroll_enabled():
return _issue(row, assetid, EndpointCredential.ENROLLED_AUTO)
return _pending(row, REASON_AWAITING_APPROVAL, assetid)
# ---------------------------------------------------------------------------
# Admin: credentials and requests
# ---------------------------------------------------------------------------
def _computers():
"""(assetid, hostname, serialnumber) for every active computer."""
Computer = _computer_model()
return (db.session.query(Computer.assetid, Computer.hostname,
Asset.serialnumber)
.join(Asset, Asset.assetid == Computer.assetid)
.filter(Asset.isactive.is_(True))
.order_by(db.func.lower(Computer.hostname))
.all())
def _pending_requests():
return (EndpointEnrollRequest.query
.filter_by(status=EndpointEnrollRequest.STATUS_PENDING)
.all())
def _pending_index(requests):
assetids = {r.matchedassetid for r in requests if r.matchedassetid}
hostnames = {(r.hostname or '').lower() for r in requests}
return assetids, hostnames
def _is_pending(assetid, hostname, index):
assetids, hostnames = index
return assetid in assetids or (hostname or '').lower() in hostnames
def _search_filter(rows, search, *keys):
if not search:
return rows
needle = search.lower()
return [row for row in rows
if any(needle in str(row.get(key) or '').lower() for key in keys)]
def _page(rows):
page, perpage = get_pagination_params()
total = len(rows)
start = (page - 1) * perpage
totalpages = (total + perpage - 1) // perpage if perpage else 0
return rows[start:start + perpage], {
'page': page, 'perpage': perpage, 'total': total,
'totalpages': totalpages, 'hasnext': page < totalpages,
'hasprev': page > 1,
}
def _settings_payload():
return {'sharedkeyallowed': agentauth.shared_key_allowed(),
'autoenroll': agentauth.auto_enroll_enabled()}
@endpoints_bp.route('/enrollment', methods=['GET'])
@require_permission('endpoints.fleet')
def get_enrollment():
"""Credential state of every computer, plus pending and recent requests."""
credentials = {c.assetid: c for c in EndpointCredential.query.all()}
pending = _pending_requests()
index = _pending_index(pending)
rows = []
for assetid, hostname, serialnumber in _computers():
credential = credentials.get(assetid)
if credential is not None and credential.revokedat is None:
state = 'enrolled'
elif _is_pending(assetid, hostname, index):
state = 'pending'
elif credential is not None:
state = 'revoked'
else:
state = 'notenrolled'
rows.append({
'assetid': assetid,
'hostname': hostname,
'serialnumber': serialnumber,
'state': state,
'credentialprefix': credential.credentialprefix if credential else None,
'issuedat': _utciso(credential.issuedat) if credential else None,
'lastusedat': _utciso(credential.lastusedat) if credential else None,
'enrolledvia': credential.enrolledvia if credential else None,
})
counts = Counter(row['state'] for row in rows)
summary = {
'enrolled': counts['enrolled'],
'notenrolled': counts['notenrolled'] + counts['revoked'],
'pending': counts['pending'],
}
state = (request.args.get('state') or '').strip().lower()
if state:
if state not in ENROLLMENT_STATES:
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Unknown state filter: {state!r}')
rows = [row for row in rows if row['state'] == state]
rows = _search_filter(rows, (request.args.get('search') or '').strip(),
'hostname', 'serialnumber')
pagerows, pagination = _page(rows)
recent = (EndpointEnrollRequest.query
.filter(EndpointEnrollRequest.status
!= EndpointEnrollRequest.STATUS_PENDING)
.order_by(EndpointEnrollRequest.lastseenat.desc())
.limit(_RECENT_REQUESTS)
.all())
requests_payload = [
r.to_dict()
for r in sorted(pending, key=lambda r: r.lastseenat or r.requestedat,
reverse=True) + recent
]
return success_response({
'settings': _settings_payload(),
'counts': summary,
'computers': pagerows,
'requests': requests_payload,
}, meta={'pagination': pagination})
@endpoints_bp.route('/enrollment/requests/<int:requestid>/approve',
methods=['POST'])
@require_role('admin')
def approve_enroll_request(requestid):
"""Bind a pending request to the asset the admin picked."""
row = db.session.get(EndpointEnrollRequest, requestid)
if row is None:
return error_response(ErrorCodes.NOT_FOUND,
f'No enroll request {requestid}', http_code=404)
if row.status == EndpointEnrollRequest.STATUS_ISSUED:
return error_response(ErrorCodes.CONFLICT,
'A credential was already issued for this request',
http_code=409)
payload = request.get_json(silent=True) or {}
assetid = payload.get('assetid')
if isinstance(assetid, bool) or not isinstance(assetid, int):
return error_response(ErrorCodes.VALIDATION_ERROR,
'assetid must be an integer')
Computer = _computer_model()
if Computer.query.filter_by(assetid=assetid).first() is None:
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Asset {assetid} is not a computer')
if agentauth.active_credential_for(assetid) is not None:
return error_response(ErrorCodes.CONFLICT,
'That computer is already enrolled; revoke its '
'credential first', http_code=409)
row.status = EndpointEnrollRequest.STATUS_APPROVED
row.matchedassetid = assetid
row.reason = None
row.decidedat = _naive_utcnow()
row.decidedby = _username()
AuditLog.log('updated', 'EndpointEnrollRequest', entityid=row.requestid,
entityname=row.hostname,
details='enroll request approved for asset {}'.format(assetid))
db.session.commit()
return success_response(row.to_dict(), message='Enroll request approved')
@endpoints_bp.route('/enrollment/requests/<int:requestid>/reject',
methods=['POST'])
@require_role('admin')
def reject_enroll_request(requestid):
row = db.session.get(EndpointEnrollRequest, requestid)
if row is None:
return error_response(ErrorCodes.NOT_FOUND,
f'No enroll request {requestid}', http_code=404)
if row.status == EndpointEnrollRequest.STATUS_ISSUED:
return error_response(ErrorCodes.CONFLICT,
'A credential was already issued for this request; '
'revoke the credential instead', http_code=409)
row.status = EndpointEnrollRequest.STATUS_REJECTED
row.decidedat = _naive_utcnow()
row.decidedby = _username()
AuditLog.log('updated', 'EndpointEnrollRequest', entityid=row.requestid,
entityname=row.hostname, details='enroll request rejected')
db.session.commit()
return success_response(row.to_dict(), message='Enroll request rejected')
@endpoints_bp.route('/enrollment/assets/<int:assetid>/revoke', methods=['POST'])
@require_role('admin')
def revoke_endpoint_credential(assetid):
"""Un-enroll a PC. Its agent gets CREDENTIAL_INVALID and enrolls again."""
credential = agentauth.active_credential_for(assetid)
if credential is None:
return error_response(ErrorCodes.NOT_FOUND,
'This computer has no active credential',
http_code=404)
credential.revokedat = _naive_utcnow()
credential.revokedby = _username()
AuditLog.log('deleted', 'EndpointCredential', entityid=assetid,
entityname=credential.credentialprefix,
details='endpoint credential {} revoked'.format(
credential.credentialprefix))
db.session.commit()
return success_response(credential.to_dict(),
message='Endpoint credential revoked')
@endpoints_bp.route('/enrollment/settings', methods=['PUT'])
@require_role('admin')
def update_enrollment_settings():
from shopdb.api import Setting
payload = request.get_json(silent=True) or {}
changes = {}
for key, settingkey in (('sharedkeyallowed', agentauth.SHARED_KEY_SETTING),
('autoenroll', agentauth.AUTO_ENROLL_SETTING)):
if key not in payload:
continue
if not isinstance(payload[key], bool):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'{key} must be true or false')
changes[key] = (settingkey, payload[key])
if not changes:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Send sharedkeyallowed and/or autoenroll')
before = _settings_payload()
for key, (settingkey, value) in changes.items():
Setting.set(settingkey, value, valuetype='boolean',
category='plugin.endpoints')
after = _settings_payload()
AuditLog.log('updated', 'Setting', entityname='endpoint enrollment',
changes={key: {'old': before[key], 'new': after[key]}
for key in changes},
details='endpoint enrollment settings changed')
db.session.commit()
return success_response(after, message='Enrollment settings saved')
# ---------------------------------------------------------------------------
# Migration of PCs already in the fleet
# ---------------------------------------------------------------------------
def _version_below(reported, target):
parsed = parse_agent_version(reported)
wanted = parse_agent_version(target)
if parsed is None or wanted is None:
return True
width = max(len(parsed), len(wanted))
return (tuple(parsed) + (0,) * (width - len(parsed))
< tuple(wanted) + (0,) * (width - len(wanted)))
def _upgrade_scriptid():
from .agentupgradescript import UPGRADE_SCRIPT_NAME
script = Script.query.filter_by(scriptname=UPGRADE_SCRIPT_NAME).first()
return script.scriptid if script is not None else None
def _store_has_package():
from . import agentstore
current = agentstore.current_version('agent')
if current is None:
return agentstore.current_path('agent') is not None
return not _version_below(current, CREDENTIAL_AGENT_VERSION)
def compute_readiness(assetids=None):
"""Migration state for every active computer (or just assetids).
Returns a list of dicts ordered by hostname: assetid, hostname,
agentversion, lastreportedat, state, blockers.
"""
computers = _computers()
hostnamecounts = Counter((hostname or '').lower()
for _assetid, hostname, _serial in computers)
if assetids is not None:
wanted = set(assetids)
computers = [row for row in computers if row[0] in wanted]
if not computers:
return []
ids = [row[0] for row in computers]
active = {c.assetid for c in EndpointCredential.query
.filter(EndpointCredential.revokedat.is_(None)).all()}
index = _pending_index(_pending_requests())
states = {s.assetid: s for s in
EndpointState.query.filter(EndpointState.assetid.in_(ids)).all()}
scriptid = _upgrade_scriptid()
upgrading = set()
if scriptid is not None:
upgrading = {row[0] for row in db.session.query(EndpointCommand.assetid)
.filter(EndpointCommand.assetid.in_(ids),
EndpointCommand.action == 'run_script',
EndpointCommand.target == f'script:{scriptid}',
EndpointCommand.status.in_(
(EndpointCommand.STATUS_PENDING,
EndpointCommand.STATUS_RUNNING)))}
haspackage = _store_has_package()
minimum = minimum_agent_version_for('run_script')
offlinecutoff = _naive_utcnow() - _OFFLINE_AFTER
result = []
for assetid, hostname, serialnumber in computers:
state = states.get(assetid)
agentversion = state.agentversion if state is not None else None
lastreportedat = state.lastreportedat if state is not None else None
blockers = []
if assetid in active:
status = 'enrolled'
elif _is_pending(assetid, hostname, index):
status = 'pending'
elif assetid in upgrading:
status = 'upgrading'
elif state is None:
status = 'notreporting'
else:
if lastreportedat is None or lastreportedat < offlinecutoff:
blockers.append(BLOCKER_OFFLINE)
if not (serialnumber or '').strip():
blockers.append(BLOCKER_NO_SERIAL)
if hostnamecounts[(hostname or '').lower()] > 1:
blockers.append(BLOCKER_DUPLICATE_HOSTNAME)
if ACTIVE_LOCK_FOR(assetid) is not None:
blockers.append(BLOCKER_LOCKED)
if not haspackage:
blockers.append(BLOCKER_NO_PACKAGE)
if minimum and not agent_meets_minimum(agentversion, minimum):
blockers.append(BLOCKER_TOO_OLD)
if not _version_below(agentversion, CREDENTIAL_AGENT_VERSION):
blockers.append(BLOCKER_AWAITING_ENROLLMENT)
if not blockers and 'run_script' not in ALLOWED_ACTIONS_FOR(
assetid=assetid):
blockers.append(BLOCKER_POLICY)
status = 'blocked' if blockers else 'ready'
result.append({
'assetid': assetid,
'hostname': hostname,
'agentversion': agentversion,
'lastreportedat': _utciso(lastreportedat),
'state': status,
'blockers': blockers,
})
return result
def _wave_progress(wave, scriptid):
members = wave.assetidlist
enrolled = set()
failed = set()
queued = set()
if members:
enrolled = {c.assetid for c in EndpointCredential.query.filter(
EndpointCredential.assetid.in_(members),
EndpointCredential.revokedat.is_(None)).all()}
if scriptid is not None:
commands = (EndpointCommand.query
.filter(EndpointCommand.assetid.in_(members),
EndpointCommand.action == 'run_script',
EndpointCommand.target == f'script:{scriptid}',
EndpointCommand.createdat >= wave.createdat)
.order_by(EndpointCommand.createdat.asc())
.all())
latest = {}
for command in commands:
queued.add(command.assetid)
latest[command.assetid] = command
failed = {assetid for assetid, command in latest.items()
if assetid not in enrolled
and command.status in (EndpointCommand.STATUS_FAILED,
EndpointCommand.STATUS_EXPIRED)}
return {
'waveid': wave.waveid,
'createdat': _utciso(wave.createdat),
'createdby': wave.createdby,
'size': len(members),
'queued': len(queued),
'enrolled': len(enrolled),
'failed': len(failed),
'done': len(enrolled) + len(failed) >= len(members),
}
@endpoints_bp.route('/enrollment/migration', methods=['GET'])
@require_permission('endpoints.fleet')
def get_enrollment_migration():
"""Per-computer readiness to move onto per-PC credentials, plus waves."""
rows = compute_readiness()
counts = Counter(row['state'] for row in rows)
state = (request.args.get('state') or '').strip().lower()
if state:
if state not in MIGRATION_STATES:
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Unknown state filter: {state!r}')
rows = [row for row in rows if row['state'] == state]
rows = _search_filter(rows, (request.args.get('search') or '').strip(),
'hostname', 'agentversion')
pagerows, pagination = _page(rows)
scriptid = _upgrade_scriptid()
waves = (EndpointMigrationWave.query
.order_by(EndpointMigrationWave.createdat.desc())
.limit(_RECENT_REQUESTS)
.all())
return success_response({
'target': CREDENTIAL_AGENT_VERSION,
'counts': {name: counts[name] for name in MIGRATION_STATES},
'computers': pagerows,
'waves': [_wave_progress(wave, scriptid) for wave in waves],
}, meta={'pagination': pagination})
def _response_message(response):
try:
body = response.get_json() or {}
except Exception: # noqa: BLE001 - reason text only
return 'enqueue refused'
error = ((body.get('data') or {}).get('error') or {})
return error.get('message') or body.get('message') or 'enqueue refused'
@endpoints_bp.route('/enrollment/migration/wave', methods=['POST'])
@require_role('admin')
def create_migration_wave():
"""Push the agent upgrade to a batch of READY PCs.
Body {"assetids": [...]} or {"next": n}. Readiness is re-checked per PC; the
agent is packaged once for the whole wave, then each PC is queued through
the same path as POST /<asset>/agentupgrade.
"""
from .routes import prepare_agent_upgrade, queue_prepared_agent_upgrade
payload = request.get_json(silent=True) or {}
requested = payload.get('assetids')
nextcount = payload.get('next')
if requested is not None:
if (not isinstance(requested, list) or not requested
or not all(isinstance(v, int) and not isinstance(v, bool)
for v in requested)):
return error_response(ErrorCodes.VALIDATION_ERROR,
'assetids must be a non-empty list of integers')
if len(requested) > _WAVE_MAXIMUM:
return error_response(ErrorCodes.VALIDATION_ERROR,
f'A wave holds at most {_WAVE_MAXIMUM} PCs')
candidates = list(dict.fromkeys(requested))
elif nextcount is not None:
if (isinstance(nextcount, bool) or not isinstance(nextcount, int)
or nextcount < 1 or nextcount > _WAVE_MAXIMUM):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'next must be an integer from 1 to {_WAVE_MAXIMUM}')
candidates = [row['assetid'] for row in compute_readiness()
if row['state'] == 'ready'][:nextcount]
else:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Send assetids or next')
readiness = {row['assetid']: row for row in compute_readiness(candidates)}
skipped = []
ready = []
for assetid in candidates:
row = readiness.get(assetid)
if row is None:
skipped.append({'assetid': assetid, 'reason': 'not a computer'})
elif row['state'] != 'ready':
reason = ', '.join(row['blockers']) if row['blockers'] else row['state']
skipped.append({'assetid': assetid, 'reason': reason})
else:
ready.append(assetid)
queued = []
waveid = None
if ready:
startedat = _naive_utcnow().replace(microsecond=0)
registration, seeded, error = prepare_agent_upgrade()
if error is not None:
return error
for assetid in ready:
response = queue_prepared_agent_upgrade(assetid, registration,
seeded, {})
if getattr(response, 'status_code', None) == 201:
queued.append(assetid)
else:
db.session.rollback()
skipped.append({'assetid': assetid,
'reason': _response_message(response)})
if queued:
wave = EndpointMigrationWave(
createdat=startedat, createdby=_username(),
assetids=','.join(str(v) for v in queued))
db.session.add(wave)
db.session.flush()
waveid = wave.waveid
AuditLog.log('created', 'EndpointMigrationWave', entityid=waveid,
entityname='agent {}'.format(registration['agentversion']),
details='migration wave queued agent upgrade for {} PC(s): {}'
.format(len(queued), wave.assetids))
db.session.commit()
return success_response(
{'waveid': waveid, 'queued': queued, 'skipped': skipped},
message='{} PC(s) queued'.format(len(queued)),
http_code=201 if queued else 200)

File diff suppressed because it is too large Load Diff

View File

@@ -1,10 +1,10 @@
"""Add fingerprint column for delta-based application updates
Revision ID: 0020
Revises: 0019
Revision ID: endpoints0020appfingerprints
Revises: endpoints0019splitpolicies
Create Date: 2026-09-14
Performance optimization: Add app_fingerprint column to endpointapplications
Performance optimization: Add appfingerprint column to endpointapplications
table to enable delta-based updates instead of delete-all-insert-all pattern.
Impact:
@@ -35,6 +35,10 @@ SAFETY FEATURES:
- Delete count verification detects silent failures
- Memory limits prevent pathological cases (>5000 apps)
- Metrics logged for monitoring
Idempotent: a database that already has the column (create_all built it from
the model, or an earlier build of this revision added it as app_fingerprint)
is left alone. 0023 renames the old spelling.
"""
from alembic import op
@@ -47,26 +51,40 @@ branch_labels = None
depends_on = None
plugin_name = 'endpoints'
TABLE = 'endpointapplications'
INDEX = 'idx_endpointapplications_fingerprint'
def _columns():
from sqlalchemy import inspect
return {c['name'] for c in inspect(op.get_bind()).get_columns(TABLE)}
def _indexes():
from sqlalchemy import inspect
return {ix['name'] for ix in inspect(op.get_bind()).get_indexes(TABLE)}
def upgrade():
# Add app_fingerprint column - nullable for backward compatibility during rollout
op.add_column('endpointapplications',
sa.Column('app_fingerprint', sa.String(64), nullable=True))
# Add UNIQUE index for fast fingerprint lookups (assetid, app_fingerprint)
# This index enables efficient delta detection: "which fingerprints exist for this asset?"
# UNIQUE constraint ensures one fingerprint per asset (delta logic requirement)
op.create_index(
'idx_endpointapplications_fingerprint',
'endpointapplications',
['assetid', 'app_fingerprint'],
unique=True
)
columns = _columns()
# Old spelling present: leave it for 0023 to rename.
if 'appfingerprint' in columns or 'app_fingerprint' in columns:
return
op.add_column(TABLE, sa.Column('appfingerprint', sa.String(64),
nullable=True))
if INDEX not in _indexes():
# Unique per asset: the delta logic keys rows on it.
op.create_index(INDEX, TABLE, ['assetid', 'appfingerprint'],
unique=True)
def downgrade():
# Remove index first
op.drop_index('idx_endpointapplications_fingerprint', 'endpointapplications')
# Remove column
op.drop_column('endpointapplications', 'app_fingerprint')
if INDEX in _indexes():
op.drop_index(INDEX, table_name=TABLE)
columns = _columns()
for name in ('appfingerprint', 'app_fingerprint'):
if name in columns:
with op.batch_alter_table(TABLE) as batch:
batch.drop_column(name)

View File

@@ -1,42 +1,26 @@
"""Add client certificate authentication columns
"""Retired: client certificate columns on assets.
Revision ID: endpoints0021clientcerts
Revises: endpoints0020appfingerprints
Create Date: 2026-09-14
This revision once added client certificate columns to the core assets table.
Client certificate authentication was replaced by per-PC endpoint credentials
(0023), so the revision is kept only so databases already stamped with it
still resolve. It does nothing; 0023 drops the columns wherever an earlier
build of this revision created them.
"""
from alembic import op
import sqlalchemy as sa
revision = 'endpoints0021clientcerts'
down_revision = 'endpoints0020appfingerprints' # Will resolve to actual revision ID
down_revision = 'endpoints0020appfingerprints'
branch_labels = None
depends_on = None
plugin_name = 'endpoints'
def upgrade():
# Add client cert columns to assets table
op.add_column('assets',
sa.Column('client_cert_fingerprint', sa.String(64), nullable=True))
op.add_column('assets',
sa.Column('cert_issued_at', sa.DateTime, nullable=True))
op.add_column('assets',
sa.Column('cert_expires_at', sa.DateTime, nullable=True))
# Add index for fast fingerprint lookup during authentication
op.create_index(
'idx_assets_cert_fingerprint',
'assets',
['client_cert_fingerprint'],
unique=False
)
pass
def downgrade():
op.drop_index('idx_assets_cert_fingerprint', 'assets')
op.drop_column('assets', 'cert_expires_at')
op.drop_column('assets', 'cert_issued_at')
op.drop_column('assets', 'client_cert_fingerprint')
pass

View File

@@ -1,12 +1,13 @@
"""Add certificate revocation columns to assets
"""Retired: client certificate revocation columns on assets.
Revision ID: endpoints0022certrevocation
Revises: endpoints0021clientcerts
Create Date: 2026-09-14
"""
from alembic import op
import sqlalchemy as sa
Kept only so databases already stamped with this revision still resolve. It
does nothing; 0023 drops the columns wherever an earlier build of this
revision created them.
"""
revision = 'endpoints0022certrevocation'
down_revision = 'endpoints0021clientcerts'
@@ -16,16 +17,8 @@ plugin_name = 'endpoints'
def upgrade():
# Add revocation tracking columns to assets table
op.add_column('assets',
sa.Column('cert_revoked_at', sa.DateTime, nullable=True,
comment='When the certificate was revoked (null = not revoked)'))
op.add_column('assets',
sa.Column('cert_revocation_reason', sa.String(255), nullable=True,
comment='Why the certificate was revoked (admin note)'))
pass
def downgrade():
op.drop_column('assets', 'cert_revocation_reason')
op.drop_column('assets', 'cert_revoked_at')
pass

View File

@@ -0,0 +1,167 @@
"""Per-PC endpoint credentials; remove the client certificate leftovers.
Revision ID: endpoints0023credentials
Revises: endpoints0022certrevocation
Create Date: 2026-09-15
Three things, each guarded against the live schema so the revision is safe on
a database built by create_all, on one that ran the earlier client certificate
builds of 0021/0022, and on a re-run:
1. Drop the client certificate columns (and their index) from the core assets
table where an earlier build of 0021/0022 created them.
2. Rename endpointapplications.app_fingerprint to appfingerprint where an
earlier build of 0020 used the underscore spelling.
3. Create endpointcredentials, endpointenrollrequests and
endpointmigrationwaves.
MySQL 5.6 safe: no JSON columns, and the enroll request lookup index uses
prefix lengths so the key stays under 767 bytes on utf8mb4.
Downgrade drops the three tables. The certificate columns are not restored:
the code that read them is gone.
"""
from alembic import op
import sqlalchemy as sa
revision = 'endpoints0023credentials'
down_revision = 'endpoints0022certrevocation'
branch_labels = None
depends_on = None
plugin_name = 'endpoints'
CREDENTIALS = 'endpointcredentials'
REQUESTS = 'endpointenrollrequests'
WAVES = 'endpointmigrationwaves'
_CERT_COLUMNS = ('client_cert_fingerprint', 'cert_issued_at', 'cert_expires_at',
'cert_revoked_at', 'cert_revocation_reason')
_APPS = 'endpointapplications'
_FINGERPRINT_INDEX = 'idx_endpointapplications_fingerprint'
def _inspector():
from sqlalchemy import inspect
return inspect(op.get_bind())
def _has_table(name):
return _inspector().has_table(name)
def _columns(table):
return {c['name'] for c in _inspector().get_columns(table)}
def _indexes(table):
return _inspector().get_indexes(table)
def _drop_certificate_columns():
if not _has_table('assets'):
return
present = _columns('assets') & set(_CERT_COLUMNS)
if not present:
return
# Any index over a certificate column goes first: 0021 named its own,
# create_all names by convention, and SQLite refuses to drop an indexed column.
for index in _indexes('assets'):
if set(index.get('column_names') or ()) & present:
op.drop_index(index['name'], table_name='assets')
for column in _CERT_COLUMNS:
if column in present:
op.drop_column('assets', column)
def _rename_fingerprint():
if not _has_table(_APPS):
return
columns = _columns(_APPS)
if 'app_fingerprint' not in columns:
return
if 'appfingerprint' in columns:
return
for index in _indexes(_APPS):
if 'app_fingerprint' in (index.get('column_names') or ()):
op.drop_index(index['name'], table_name=_APPS)
op.alter_column(_APPS, 'app_fingerprint', new_column_name='appfingerprint',
existing_type=sa.String(64), existing_nullable=True)
op.create_index(_FINGERPRINT_INDEX, _APPS, ['assetid', 'appfingerprint'],
unique=True)
def upgrade():
_drop_certificate_columns()
_rename_fingerprint()
if not _has_table(CREDENTIALS):
op.create_table(
CREDENTIALS,
sa.Column('credentialid', sa.Integer(), nullable=False),
sa.Column('assetid', sa.Integer(), nullable=False),
sa.Column('credentialhash', sa.CHAR(length=64), nullable=False),
sa.Column('credentialprefix', sa.String(length=12), nullable=False),
sa.Column('enrolledvia', sa.String(length=16), nullable=False),
sa.Column('issuedat', sa.DateTime(), nullable=False),
sa.Column('lastusedat', sa.DateTime(), nullable=True),
sa.Column('revokedat', sa.DateTime(), nullable=True),
sa.Column('revokedby', sa.String(length=64), nullable=True),
sa.ForeignKeyConstraint(['assetid'], ['assets.assetid'],
ondelete='CASCADE'),
sa.PrimaryKeyConstraint('credentialid'),
sa.UniqueConstraint('assetid'),
sa.UniqueConstraint('credentialhash'),
)
if not _has_table(REQUESTS):
op.create_table(
REQUESTS,
sa.Column('requestid', sa.Integer(), nullable=False),
sa.Column('hostname', sa.String(length=255), nullable=False),
sa.Column('serialnumber', sa.String(length=128), nullable=False),
sa.Column('deviceid', sa.String(length=64), nullable=True),
sa.Column('enrollnonce', sa.String(length=128), nullable=False),
sa.Column('agentversion', sa.String(length=32), nullable=True),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('reason', sa.String(length=64), nullable=True),
sa.Column('matchedassetid', sa.Integer(), nullable=True),
sa.Column('sourceip', sa.String(length=45), nullable=True),
sa.Column('requestedat', sa.DateTime(), nullable=False),
sa.Column('lastseenat', sa.DateTime(), nullable=False),
sa.Column('decidedat', sa.DateTime(), nullable=True),
sa.Column('decidedby', sa.String(length=64), nullable=True),
sa.ForeignKeyConstraint(['matchedassetid'], ['assets.assetid'],
ondelete='SET NULL'),
sa.PrimaryKeyConstraint('requestid'),
)
existing = {tuple(ix.get('column_names') or ()) for ix in _indexes(REQUESTS)}
if ('hostname', 'serialnumber') not in existing:
op.create_index('idx_endpointenrollrequests_hostserial', REQUESTS,
['hostname', 'serialnumber'],
mysql_length={'hostname': 128, 'serialnumber': 63})
if not _has_table(WAVES):
op.create_table(
WAVES,
sa.Column('waveid', sa.Integer(), nullable=False),
sa.Column('createdat', sa.DateTime(), nullable=False),
sa.Column('createdby', sa.String(length=64), nullable=True),
sa.Column('assetids', sa.Text(), nullable=False),
sa.Column('notes', sa.String(length=255), nullable=True),
sa.PrimaryKeyConstraint('waveid'),
)
def downgrade():
if _has_table(WAVES):
op.drop_table(WAVES)
if _has_table(REQUESTS):
for index in _indexes(REQUESTS):
if index['name'] == 'idx_endpointenrollrequests_hostserial':
op.drop_index(index['name'], table_name=REQUESTS)
op.drop_table(REQUESTS)
if _has_table(CREDENTIALS):
op.drop_table(CREDENTIALS)

View File

@@ -6,6 +6,9 @@ from .endpoint import (
)
from .monitor import EndpointMonitor
from .operatorreport import OperatorReport
from .credential import (
EndpointCredential, EndpointEnrollRequest, EndpointMigrationWave,
)
from .health import (
EndpointHealthState, EndpointHealthTransition, EndpointMonitorImpact,
)
@@ -14,5 +17,6 @@ __all__ = [
'EndpointState', 'EndpointEvent', 'EndpointCommand', 'EndpointApplication',
'EndpointLock', 'EndpointLogCollection',
'EndpointMonitor', 'OperatorReport',
'EndpointCredential', 'EndpointEnrollRequest', 'EndpointMigrationWave',
'EndpointHealthState', 'EndpointHealthTransition', 'EndpointMonitorImpact',
]

View File

@@ -0,0 +1,148 @@
"""Per-PC endpoint credentials, enrollment requests and migration waves.
EndpointCredential
One row per asset. Holds only the sha256 of the secret the
agent was issued, plus a short display prefix. Revoking sets
revokedat; re-issuing after a revoke UPDATES the same row, so
an asset never carries two credentials.
EndpointEnrollRequest
One row per (hostname, serial number) that asked to enroll. The
agent's enrollment nonce is recorded on first sight and is what
an admin approval is bound to, so a second machine claiming the
same hostname and serial cannot collect the approved credential.
EndpointMigrationWave
An admin action that pushed the agent upgrade to a set of PCs.
Progress is computed live from credentials and commands; only
the membership is stored.
"""
from datetime import datetime
from shopdb.api import db
def _utciso(value):
return value.isoformat() + 'Z' if value else None
class EndpointCredential(db.Model):
"""The credential issued to one endpoint (hash only)."""
__tablename__ = 'endpointcredentials'
ENROLLED_AUTO = 'auto'
ENROLLED_APPROVED = 'approved'
credentialid = db.Column(db.Integer, primary_key=True)
assetid = db.Column(
db.Integer,
db.ForeignKey('assets.assetid', ondelete='CASCADE'),
nullable=False,
unique=True,
)
credentialhash = db.Column(db.CHAR(64), nullable=False, unique=True)
credentialprefix = db.Column(db.String(12), nullable=False)
enrolledvia = db.Column(db.String(16), nullable=False)
issuedat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
lastusedat = db.Column(db.DateTime, nullable=True)
revokedat = db.Column(db.DateTime, nullable=True)
revokedby = db.Column(db.String(64), nullable=True)
@property
def isactive(self):
return self.revokedat is None
def to_dict(self):
return {
'credentialid': self.credentialid,
'assetid': self.assetid,
'credentialprefix': self.credentialprefix,
'enrolledvia': self.enrolledvia,
'issuedat': _utciso(self.issuedat),
'lastusedat': _utciso(self.lastusedat),
'revokedat': _utciso(self.revokedat),
'revokedby': self.revokedby,
}
def __repr__(self):
return f'<EndpointCredential asset={self.assetid} {self.credentialprefix}>'
class EndpointEnrollRequest(db.Model):
"""A PC asking for a credential."""
__tablename__ = 'endpointenrollrequests'
STATUS_PENDING = 'pending'
STATUS_APPROVED = 'approved'
STATUS_REJECTED = 'rejected'
STATUS_ISSUED = 'issued'
requestid = db.Column(db.Integer, primary_key=True)
hostname = db.Column(db.String(255), nullable=False)
serialnumber = db.Column(db.String(128), nullable=False)
deviceid = db.Column(db.String(64), nullable=True)
enrollnonce = db.Column(db.String(128), nullable=False)
agentversion = db.Column(db.String(32), nullable=True)
status = db.Column(db.String(16), nullable=False, default=STATUS_PENDING)
reason = db.Column(db.String(64), nullable=True)
matchedassetid = db.Column(
db.Integer,
db.ForeignKey('assets.assetid', ondelete='SET NULL'),
nullable=True,
)
sourceip = db.Column(db.String(45), nullable=True)
requestedat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
lastseenat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
decidedat = db.Column(db.DateTime, nullable=True)
decidedby = db.Column(db.String(64), nullable=True)
# Prefix lengths keep the key under 767 bytes on utf8mb4 (4 bytes a char).
__table_args__ = (
db.Index('idx_endpointenrollrequests_hostserial', 'hostname',
'serialnumber',
mysql_length={'hostname': 128, 'serialnumber': 63}),
)
def to_dict(self):
return {
'requestid': self.requestid,
'hostname': self.hostname,
'serialnumber': self.serialnumber,
'deviceid': self.deviceid,
'agentversion': self.agentversion,
'reason': self.reason,
'status': self.status,
'matchedassetid': self.matchedassetid,
'requestedat': _utciso(self.requestedat),
'lastseenat': _utciso(self.lastseenat),
'decidedat': _utciso(self.decidedat),
'decidedby': self.decidedby,
'sourceip': self.sourceip,
}
def __repr__(self):
return f'<EndpointEnrollRequest {self.hostname!r} {self.status}>'
class EndpointMigrationWave(db.Model):
"""One batch of agent upgrades pushed to move PCs onto credentials."""
__tablename__ = 'endpointmigrationwaves'
waveid = db.Column(db.Integer, primary_key=True)
createdat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
createdby = db.Column(db.String(64), nullable=True)
# Comma list of asset ids. TEXT, not JSON: MySQL 5.6.
assetids = db.Column(db.Text, nullable=False, default='')
notes = db.Column(db.String(255), nullable=True)
@property
def assetidlist(self):
return [int(part) for part in (self.assetids or '').split(',')
if part.strip().isdigit()]
def __repr__(self):
return f'<EndpointMigrationWave {self.waveid}>'

View File

@@ -468,11 +468,11 @@ class EndpointApplication(db.Model):
# publisher. Allows efficient comparison to detect changes without comparing all
# columns. Computed by agent and sent in payload. Nullable for backward compatibility
# during migration - backfill script will populate existing rows.
app_fingerprint = db.Column(db.String(64), nullable=True)
appfingerprint = db.Column(db.String(64), nullable=True)
__table_args__ = (
db.Index('idx_endpointapplications_assetname', 'assetid', 'displayname'),
db.Index('idx_endpointapplications_fingerprint', 'assetid', 'app_fingerprint', unique=True),
db.Index('idx_endpointapplications_fingerprint', 'assetid', 'appfingerprint', unique=True),
)
def __repr__(self):

View File

@@ -20,7 +20,7 @@ from datetime import datetime, timezone
from pathlib import Path
from typing import Dict, List, Optional, Type
from flask import Flask, Blueprint
from flask import Flask, Blueprint, g
from shopdb.plugins.base import BasePlugin, PluginMeta
from shopdb.api import db
@@ -298,6 +298,25 @@ class EndpointsPlugin(BasePlugin):
'key is provisioned so revocation is a real kill '
'switch.',
},
{
'key': 'plugin.endpoints.sharedkeyallowed',
'value': 'true',
'valuetype': 'boolean',
'category': 'plugin.endpoints',
'description': 'Accept the shared fleet collector key from PCs '
'that have not enrolled. Enrolled PCs are always '
'refused it. Turn off once every PC has a '
'per-PC credential.',
},
{
'key': 'plugin.endpoints.autoenroll',
'value': 'true',
'valuetype': 'boolean',
'category': 'plugin.endpoints',
'description': 'Issue a per-PC credential without admin '
'approval when the hostname and serial number '
'match exactly one computer.',
},
{
'key': 'plugin.endpoints.agentkeyprovisioned',
'value': 'false',
@@ -351,6 +370,15 @@ class EndpointsPlugin(BasePlugin):
'Windows trusted roots.',
'position': 29,
},
{
'group': 'Endpoints',
'to': '/settings/endpoints/enrollment',
'icon': 'key',
'title': 'Enrollment',
'description': 'Move agents to per-PC credentials, approve '
'enrollment requests, and retire the shared key.',
'position': 30,
},
]
def get_navigation_items(self) -> List[Dict]:
@@ -804,9 +832,18 @@ class EndpointsPlugin(BasePlugin):
# ---- ADR-006 collector contract -------------------------------------
def resolve_agent_credential(self, secret: str) -> Optional[int]:
from .api import agentauth
return agentauth.resolve_credential(secret)
def agent_shared_key_refused(self, hostname: str) -> bool:
from .api import agentauth
return agentauth.shared_key_refused_for(hostname)
def get_collector_schema(self) -> Optional[Dict]:
return {
'identityfield': 'hostname',
'acceptsagentcredential': True,
'fields': {
'type': 'object',
'required': ['hostname'],
@@ -1086,7 +1123,7 @@ class EndpointsPlugin(BasePlugin):
displayname=displayname[:255],
displayversion=_clamp(item.get('displayversion'), 128),
publisher=_clamp(item.get('publisher'), 255),
app_fingerprint=None, # Agent didn't send it
appfingerprint=None, # Agent didn't send it
))
stored += 1
@@ -1153,7 +1190,7 @@ class EndpointsPlugin(BasePlugin):
displayname=app_data['displayname'],
displayversion=app_data['displayversion'],
publisher=app_data['publisher'],
app_fingerprint=fp,
appfingerprint=fp,
))
stored += 1
@@ -1164,14 +1201,14 @@ class EndpointsPlugin(BasePlugin):
)
return stored
current_fps = {app.app_fingerprint for app in current_apps
if app.app_fingerprint} # Skip NULL fingerprints from migration
current_fps = {app.appfingerprint for app in current_apps
if app.appfingerprint} # Skip NULL fingerprints from migration
# TRANSITION: If we have apps without fingerprints, this is the first report
# after agent upgrade. Delete all NULL-fingerprint apps to prevent duplicates.
# SAFETY: Validate incoming apps BEFORE deleting legacy apps to prevent
# leaving asset with zero apps if transaction aborts.
null_fp_apps = [app for app in current_apps if app.app_fingerprint is None]
null_fp_apps = [app for app in current_apps if app.appfingerprint is None]
if null_fp_apps:
if not incoming_fps:
warnings.append(
@@ -1192,7 +1229,7 @@ class EndpointsPlugin(BasePlugin):
if to_remove:
deleted_count = EndpointApplication.query.filter(
EndpointApplication.assetid == assetid,
EndpointApplication.app_fingerprint.in_(list(to_remove))
EndpointApplication.appfingerprint.in_(list(to_remove))
).delete(synchronize_session=False)
# SAFETY: Verify delete count matches expected. Log as WARNING
@@ -1215,7 +1252,7 @@ class EndpointsPlugin(BasePlugin):
displayname=app_data['displayname'],
displayversion=app_data['displayversion'],
publisher=app_data['publisher'],
app_fingerprint=fp,
appfingerprint=fp,
))
# SAFETY: Flush to detect constraint violations before continuing
@@ -1240,6 +1277,14 @@ class EndpointsPlugin(BasePlugin):
if not hostname:
raise ValueError('hostname is required')
from .api import agentauth
if agentauth.is_credential_caller():
# A credentialed agent reports only for its own asset, and never
# adopts: its asset already exists.
if agentauth.resolve_hostname(hostname) != g.agentassetid:
raise PermissionError(
'The payload names a different endpoint than the credential')
computer = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.first())
@@ -1270,21 +1315,6 @@ class EndpointsPlugin(BasePlugin):
assetid = computer.assetid
# Authorization: if authenticated via client certificate, verify the
# certificate's asset matches the payload's hostname asset. Without this
# check an agent with a valid cert for asset A can submit telemetry for
# asset B by changing the hostname field.
from flask import g
if hasattr(g, 'cert_authenticated_asset_id'):
if g.cert_authenticated_asset_id != assetid:
self.logger.warning(
'[collector] AUTHORIZATION DENIED: Certificate authenticates '
'asset %d but payload hostname %s resolves to asset %d',
g.cert_authenticated_asset_id, hostname, assetid)
raise PermissionError(
f'Certificate authenticates asset {g.cert_authenticated_asset_id} '
f'but payload reports for asset {assetid}')
state = EndpointState.query.filter_by(assetid=assetid).first()
action = 'updated'
if state is None:
@@ -1510,12 +1540,18 @@ class EndpointsPlugin(BasePlugin):
# Fail-open, matching the reconciler above: a health error must never
# fail an otherwise-good check-in. The previous verdict simply stands,
# which is correct - a failed evaluation is not evidence of health.
#
# SAVEPOINT: a failed flush (IntegrityError and friends) otherwise
# leaves the session in a rolled-back-needed state, and the caught
# exception would still sink the outer commit of the whole check-in.
# Rolling back only the savepoint keeps the telemetry flushed above.
try:
from .services.health import evaluate_endpoint_health
evaluate_endpoint_health(
assetid, state,
computertypeid=getattr(computer, 'computertypeid', None))
db.session.flush()
with db.session.begin_nested():
evaluate_endpoint_health(
assetid, state,
computertypeid=getattr(computer, 'computertypeid', None))
db.session.flush()
except Exception:
logger.exception(
'health evaluation failed for asset %s; leaving verdict '

View File

@@ -11,7 +11,7 @@ from flask_jwt_extended import jwt_required
from shopdb.api import db, success_response, error_response, paginated_response, ErrorCodes, get_pagination_params, paginate_query
from ..models import Notification, NotificationType
from ..models.notification import _utc_iso
from ..models.notification import _utc_iso, ALERT_SCOPE_KEYS
from shopdb.api import require_permission, Setting, intarg
@@ -405,6 +405,69 @@ def get_notification(notification_id: int):
return success_response(n.to_dict())
def _clean_alert_fields(data):
"""Validate and normalise alertfleet / alertscope / alerttimeout.
Returns (fields, error). fields holds only the keys present in data, with
null coerced to the column default. alertscope must be a JSON object (or a
string holding one) whose keys the endpoint matcher understands; empty or
null means the whole fleet.
"""
import json
fields = {}
if 'alertfleet' in data:
value = data['alertfleet']
if value is None:
value = False
if not isinstance(value, bool):
return None, 'alertfleet must be true or false'
fields['alertfleet'] = value
if 'alerttimeout' in data:
value = data['alerttimeout']
if value is None or value == '':
value = 0
if isinstance(value, bool):
return None, 'alerttimeout must be a whole number of seconds'
try:
value = int(value)
except (TypeError, ValueError):
return None, 'alerttimeout must be a whole number of seconds'
if value < 0:
return None, 'alerttimeout must not be negative'
fields['alerttimeout'] = value
if 'alertscope' in data:
scope = data['alertscope']
if isinstance(scope, str):
if not scope.strip():
scope = None
else:
try:
scope = json.loads(scope)
except ValueError:
return None, 'alertscope must be a JSON object'
if scope is not None:
if not isinstance(scope, dict):
return None, 'alertscope must be a JSON object'
unknown = sorted(set(scope) - ALERT_SCOPE_KEYS)
if unknown:
return None, (f'alertscope has unknown key(s): {", ".join(unknown)}. '
f'Allowed: {", ".join(sorted(ALERT_SCOPE_KEYS))}')
for key, wanted in scope.items():
values = wanted if isinstance(wanted, list) else [wanted]
if any(v is not None and not isinstance(v, (str, int, float))
or isinstance(v, bool) for v in values):
return None, (f'alertscope.{key} must be a string or a list '
f'of strings')
if not scope:
scope = None
fields['alertscope'] = scope
return fields, None
@notifications_bp.route('', methods=['POST'])
@require_permission('notifications.create')
def create_notification():
@@ -419,6 +482,10 @@ def create_notification():
if not notification_text:
return error_response(ErrorCodes.VALIDATION_ERROR, 'notification/message is required')
alert_fields, alert_error = _clean_alert_fields(data)
if alert_error:
return error_response(ErrorCodes.VALIDATION_ERROR, alert_error)
# Parse dates
starttime = datetime.now(timezone.utc).replace(tzinfo=None)
if data.get('starttime') or data.get('startdate'):
@@ -456,9 +523,9 @@ def create_notification():
isshopfloor=data.get('isshopfloor', False),
employeesso=data.get('employeesso'),
employeename=data.get('employeename'),
alertfleet=data.get('alertfleet', False),
alertscope=data.get('alertscope'),
alerttimeout=data.get('alerttimeout', 0)
alertfleet=alert_fields.get('alertfleet', False),
alertscope=alert_fields.get('alertscope'),
alerttimeout=alert_fields.get('alerttimeout', 0)
)
db.session.add(n)
@@ -507,12 +574,12 @@ def update_notification(notification_id: int):
n.employeesso = data['employeesso']
if 'employeename' in data:
n.employeename = data['employeename']
if 'alertfleet' in data:
n.alertfleet = data['alertfleet']
if 'alertscope' in data:
n.alertscope = data['alertscope']
if 'alerttimeout' in data:
n.alerttimeout = data['alerttimeout']
alert_fields, alert_error = _clean_alert_fields(data)
if alert_error:
db.session.rollback()
return error_response(ErrorCodes.VALIDATION_ERROR, alert_error)
for field, value in alert_fields.items():
setattr(n, field, value)
# Parse and update dates
if 'starttime' in data or 'startdate' in data:

View File

@@ -7,6 +7,12 @@ from shopdb.api import db
_DEFAULT_TZ = 'America/New_York'
# Keys a fleet alert's alertscope may use. Must stay equal to the set the
# endpoints matcher (_ALERT_SCOPE_KEYS in plugins/endpoints/api/routes.py)
# understands; a test pins the two together.
ALERT_SCOPE_KEYS = frozenset({'hostname', 'assettype', 'computertype',
'location', 'businessunit'})
def _site_zone():
"""Site-configured IANA zone (settings key site_timezone) for calendar day
@@ -146,9 +152,12 @@ class Notification(db.Model):
# ~11 people are listed.
employeesso = db.Column(db.Text, nullable=True)
employeename = db.Column(db.Text, nullable=True)
alertfleet = db.Column(db.Boolean, default=False)
# Matches migration 0006: NOT NULL with server defaults.
alertfleet = db.Column(db.Boolean, nullable=False, default=False,
server_default=db.false())
alertscope = db.Column(db.JSON, nullable=True)
alerttimeout = db.Column(db.Integer, nullable=True)
alerttimeout = db.Column(db.Integer, nullable=False, default=0,
server_default='0')
# Relationships
notificationtype = db.relationship('NotificationType', backref='notifications')

View File

@@ -7,7 +7,7 @@ import re
from typing import List, Dict, Optional, Type
from flask import Flask, Blueprint
from flask import Flask, Blueprint, g
import click
from shopdb.plugins.base import BasePlugin, PluginMeta
@@ -439,6 +439,7 @@ class PrintersPlugin(BasePlugin):
"""
return {
'identityfield': 'hostname',
'acceptsagentcredential': True,
'fields': {
'type': 'object',
'required': ['hostname', 'queues'],
@@ -551,6 +552,11 @@ class PrintersPlugin(BasePlugin):
# the read paths, and hostname stays the identity that the replace
# keys on, so an unresolved host still records everything it reported.
assetid = self._observed_assetid(hostname, warnings)
# A per-PC credential reports for its own PC only.
if (getattr(g, 'agentauth', None) == 'credential'
and assetid != getattr(g, 'agentassetid', None)):
raise PermissionError(
'The payload names a different endpoint than the credential')
# Case-folded on both sides: one script sends COMPUTERNAME uppercase
# and another the lowercase FQDN, and MySQL forgives that while SQLite

View File

@@ -1,26 +0,0 @@
#!/usr/bin/env python
"""Development server with SQLite support (patches engine options)."""
import os
import sys
# Set environment for development with SQLite
os.environ['FLASK_ENV'] = 'development'
os.environ['DATABASE_URL'] = f"sqlite:///{os.path.abspath('instance/shopdb.db')}"
# Import and patch config before app creation
from shopdb.config import DevelopmentConfig
from sqlalchemy.pool import StaticPool
# Override engine options for SQLite (no charset parameter)
DevelopmentConfig.SQLALCHEMY_ENGINE_OPTIONS = {
'connect_args': {'check_same_thread': False},
'poolclass': StaticPool,
}
# Now import and run the app
from shopdb import create_app
app = create_app('development')
if __name__ == '__main__':
app.run(debug=True, host='127.0.0.1', port=5000)

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Backfill app_fingerprint column for existing EndpointApplication records.
"""Backfill appfingerprint column for existing EndpointApplication records.
Run this ONCE after deploying the 0020_app_delta_fingerprints migration and
before deploying the updated C# agent that sends fingerprints. This populates
@@ -20,6 +20,13 @@ from pathlib import Path
# Add project root to Python path
sys.path.insert(0, str(Path(__file__).parent.parent))
from dotenv import load_dotenv
# Load .env before anything reads config: a plain `python scripts/...` run
# otherwise falls back to the hardcoded root@localhost URL and could write the
# wrong database.
load_dotenv(Path(__file__).resolve().parent.parent / '.env')
from shopdb.extensions import db
from plugins.endpoints.models import EndpointApplication
@@ -50,103 +57,122 @@ def compute_fingerprint(displayname: str, displayversion: str, publisher: str) -
def verify_fingerprint_algorithm():
"""Verify backend fingerprint algorithm matches expected behavior.
Uses test vectors to ensure consistency with agent implementation.
Any mismatch indicates algorithm drift and will cause delta logic to fail.
Fixed test vectors: SHA256 of the literal 'name|version|publisher' strings,
which is what the agent's ComputeApplicationFingerprint hashes. Any
mismatch means compute_fingerprint drifted from the agent, which would make
the delta logic delete and re-add every app on every check-in.
"""
print("Verifying fingerprint algorithm...")
# Test vectors: known inputs and their expected SHA256 outputs
# These verify the algorithm matches the C# agent implementation
test_cases = [
{
'name': 'Google Chrome',
'version': '117.0.5938.132',
'publisher': 'Google LLC',
# SHA256("Google Chrome|117.0.5938.132|Google LLC")
'expected_fp': None # Prints computed value for verification
},
{
'name': 'Microsoft Office',
'version': '16.0.16827.20166',
'publisher': 'Microsoft Corporation',
# SHA256("Microsoft Office|16.0.16827.20166|Microsoft Corporation")
'expected_fp': None # Prints computed value for verification
},
{
'name': 'Test App No Version',
'version': None,
'publisher': 'Test Publisher',
# SHA256("Test App No Version||Test Publisher")
'expected_fp': None # Prints computed value for verification
},
{
'name': 'Test App Empty Strings',
'version': '',
'publisher': '',
# SHA256("Test App Empty Strings||")
'expected_fp': None # Prints computed value for verification
},
('Google Chrome', '117.0.5938.132', 'Google LLC',
'a22de9ae7e641954872208b21aa52ac948898b78bcc8d7de5746af3a324f49ae'),
('Microsoft Office', '16.0.16827.20166', 'Microsoft Corporation',
'a180bfd16f34c52c94d01317044f7d3112bd9b88c6be570ee5cca4eff9199ae4'),
('Test App No Version', None, 'Test Publisher',
'aae1e046f90b39da80b2356713d8e6dc8579746d378ce7bc515800a4fbb85cda'),
('Test App Empty Strings', '', '',
'1a29cd2e3bd856ac3143d2675376b92eedf1563f5e8eb78f235d252943580ceb'),
]
all_passed = True
for tc in test_cases:
computed = compute_fingerprint(tc['name'], tc['version'], tc['publisher'])
# Validate format
if len(computed) != 64 or not all(c in '0123456789abcdef' for c in computed):
print(f" FAIL INVALID FORMAT: {tc['name']}")
print(f" Got: {computed}")
all_passed = False
continue
# Check against expected (if we have one)
if tc['expected_fp']:
if computed == tc['expected_fp']:
print(f" OK {tc['name']}: {computed[:16]}...")
else:
print(f" FAIL MISMATCH: {tc['name']}")
print(f" Expected: {tc['expected_fp']}")
print(f" Got: {computed}")
all_passed = False
failures = []
for name, version, publisher, expected in test_cases:
computed = compute_fingerprint(name, version, publisher)
if computed == expected:
print(f" OK {name}: {computed[:16]}...")
else:
# First run - show computed value for test vector creation
print(f" ? {tc['name']}: {computed[:16]}... (computed, no expected value)")
print(f" FAIL MISMATCH: {name}")
print(f" Expected: {expected}")
print(f" Got: {computed}")
failures.append(name)
if not all_passed:
if failures:
raise ValueError(
"Fingerprint algorithm verification FAILED! "
"Backend algorithm does not match expected behavior. "
"This will cause delta logic to delete+re-add all apps on every check-in."
"Fingerprint algorithm verification FAILED for: "
+ ", ".join(failures)
+ ". Backend algorithm does not match the agent; the delta logic "
"would delete+re-add all apps on every check-in."
)
print("OK Fingerprint algorithm verified\n")
return True
def _fingerprint_batch(apps):
"""Assign fingerprints to one batch, removing duplicate (assetid, fingerprint) rows.
The unique index on (assetid, appfingerprint) means two rows for the same
install on one asset cannot both carry the fingerprint. Of each duplicate
set the lowest endpointapplicationid is kept (fingerprinted rows already in
the table included) and the rest are deleted.
Returns the number of duplicate rows deleted.
"""
duplicates = 0
claimed = {}
for app in sorted(apps, key=lambda row: row.endpointapplicationid):
fingerprint = compute_fingerprint(app.displayname, app.displayversion,
app.publisher)
key = (app.assetid, fingerprint)
holder = claimed.get(key)
if holder is None:
holder = (EndpointApplication.query
.filter(EndpointApplication.assetid == app.assetid,
EndpointApplication.appfingerprint == fingerprint,
EndpointApplication.endpointapplicationid
!= app.endpointapplicationid)
.first())
if holder is None:
app.appfingerprint = fingerprint
claimed[key] = app
elif holder.endpointapplicationid < app.endpointapplicationid:
db.session.delete(app)
duplicates += 1
else:
# An already-fingerprinted row with a HIGHER id: drop it and let the
# older row carry the fingerprint. Flush the delete first so the
# unique index is free before the update lands.
db.session.delete(holder)
db.session.flush()
duplicates += 1
app.appfingerprint = fingerprint
claimed[key] = app
return duplicates
def backfill_fingerprints(batch_size: int = 1000, dry_run: bool = False):
"""Backfill fingerprints for existing EndpointApplication records.
Each batch commits on its own. A batch that fails is rolled back and the run
stops, so nothing half-applied is left in the session.
Args:
batch_size: Number of records to update per commit
dry_run: If True, show what would be done without making changes
Returns:
dict with processed, duplicatesremoved and failed counts.
"""
# SAFETY: Verify algorithm before touching database
verify_fingerprint_algorithm()
# Count total records needing backfill
summary = {'processed': 0, 'duplicatesremoved': 0, 'failed': 0}
total = EndpointApplication.query.filter(
EndpointApplication.app_fingerprint.is_(None)
EndpointApplication.appfingerprint.is_(None)
).count()
if total == 0:
print("No records need backfilling (all have fingerprints).")
return
return summary
print(f"Found {total:,} records without fingerprints")
if dry_run:
print("\nDRY RUN MODE - showing sample records:")
sample = EndpointApplication.query.filter(
EndpointApplication.app_fingerprint.is_(None)
EndpointApplication.appfingerprint.is_(None)
).limit(5).all()
for app in sample:
@@ -154,49 +180,54 @@ def backfill_fingerprints(batch_size: int = 1000, dry_run: bool = False):
print(f" {app.displayname:50s} -> {fp[:16]}...")
print(f"\nWould backfill {total:,} records in batches of {batch_size}")
return
return summary
print(f"Backfilling in batches of {batch_size}...")
processed = 0
batch = 0
while True:
# Fetch next batch
apps = (EndpointApplication.query
.filter(EndpointApplication.app_fingerprint.is_(None))
.filter(EndpointApplication.appfingerprint.is_(None))
.order_by(EndpointApplication.endpointapplicationid)
.limit(batch_size)
.all())
if not apps:
break # No more records
# Update fingerprints
for app in apps:
app.app_fingerprint = compute_fingerprint(
app.displayname,
app.displayversion,
app.publisher
)
# Commit batch
db.session.commit()
break
batch += 1
processed += len(apps)
print(f" Batch {batch}: {processed:,} / {total:,} ({processed*100//total}%)")
try:
duplicates = _fingerprint_batch(apps)
db.session.commit()
except Exception as error:
db.session.rollback()
summary['failed'] = len(apps)
print(f" Batch {batch} FAILED and was rolled back: {error}")
break
summary['processed'] += len(apps)
summary['duplicatesremoved'] += duplicates
print(f" Batch {batch}: {summary['processed']:,} / {total:,} "
f"({summary['processed'] * 100 // total}%), "
f"{duplicates} duplicate(s) removed")
print(f"\nBackfill finished. Processed {summary['processed']:,} records; "
f"removed {summary['duplicatesremoved']:,} duplicate row(s).")
if summary['failed']:
print("A batch failed; fix the error above and re-run (completed batches "
"are kept, the run resumes from the remaining null rows).")
return summary
print(f"\nBackfill complete! Updated {processed:,} records.")
print("\nNext steps:")
print("1. Deploy updated C# agent that computes and sends fingerprints")
print("2. Monitor collector warnings for agents without fingerprint support")
print("3. After all agents updated, consider making app_fingerprint NOT NULL")
print("3. After all agents updated, consider making appfingerprint NOT NULL")
return summary
if __name__ == '__main__':
import argparse
parser = argparse.ArgumentParser(description='Backfill app_fingerprint column')
parser = argparse.ArgumentParser(description='Backfill appfingerprint column')
parser.add_argument('--dry-run', action='store_true',
help='Show what would be done without making changes')
parser.add_argument('--batch-size', type=int, default=1000,
@@ -209,10 +240,12 @@ if __name__ == '__main__':
from shopdb import create_app
app = create_app()
with app.app_context():
backfill_fingerprints(
result = backfill_fingerprints(
batch_size=args.batch_size,
dry_run=args.dry_run
)
except Exception as e:
print(f"Error: {e}")
sys.exit(1)
if result['failed']:
sys.exit(1)

View File

@@ -25,7 +25,9 @@ USE_GIT_LS_FILES=${USE_GIT_LS_FILES:-1}
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
# Filter out paths we don't want to scan even if tracked
GIT_FILES_FILTER="grep -v '^migrations/versions/' | grep -v '^frontend/dist/' | grep -v '^staticdocs/' | grep -v '\.plugins-staged/'"
# NUL-delimited end to end (git ls-files -z | grep -z | xargs -0) so a path
# with a space or newline is scanned as one file, not split into several.
GIT_FILES_FILTER="grep -zv '^migrations/versions/' | grep -zv '^frontend/dist/' | grep -zv '^staticdocs/' | grep -zv '\.plugins-staged/'"
else
# Fallback to recursive grep with excludes for non-git directories
GIT_FILES_FILTER="cat"
@@ -113,7 +115,9 @@ drop_scratch() { grep -v '^\./devnotes/' || true; }
# lists it sat beside.
drop_generated() {
# Match with or without leading ./ for git ls-files compatibility
grep -vP '^(\./)?(.*/)?(findings\.sarif|report\.md|scan\.log|\.env|.*\.standalone\.js|.*\.min\.js):' || true
# [^:]* rather than .*: on a long minified hit line the greedy form blows
# PCRE's backtracking limit, and grep then errors instead of filtering.
grep -vP '^(\./)?([^:]*/)?(findings\.sarif|report\.md|scan\.log|\.env|[^:/]*\.standalone\.js|[^:/]*\.min\.js):' || true
}
# CONTRIBUTING.md documents the conventions by SHOWING violations as examples,
@@ -153,10 +157,10 @@ echo "==> Checking for non-ASCII characters..."
# wj wrote this as --binary-files=without-match, which is the same behaviour
# spelled long-hand; -I is kept for consistency with the rest of the file.
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
NON_ASCII=$(git ls-files \
NON_ASCII=$(git ls-files -z \
'*.py' '*.vue' '*.js' '*.ts' '*.json' '*.md' '*.yaml' '*.yml' \
| eval "$GIT_FILES_FILTER" \
| xargs -r grep -PnI '[^\x00-\x7F]' 2>/dev/null \
| xargs -0 -r grep -PnI '[^\x00-\x7F]' 2>/dev/null \
| drop_scratch | drop_generated || true)
else
NON_ASCII=$(grep -rPnI '[^\x00-\x7F]' "${EXCLUDES[@]}" "${INCLUDES_ALL[@]}" . 2>/dev/null | drop_scratch | drop_generated || true)
@@ -177,11 +181,14 @@ echo "==> Checking for banned shorthand (standalone)..."
# Exclude C# client code (.cs files) - they're not Python and have different conventions
for word in cfg ctx mgr req res; do
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
HITS=$(git ls-files 'shopdb/**/*.py' 'shopdb/**/*.vue' 'shopdb/**/*.js' 'shopdb/**/*.ts' \
'plugins/**/*.py' 'plugins/**/*.vue' 'plugins/**/*.js' 'plugins/**/*.ts' \
'frontend/src/**/*.py' 'frontend/src/**/*.vue' 'frontend/src/**/*.js' 'frontend/src/**/*.ts' \
| grep -v '/tests\?/' | grep -v '\.spec\.js' | grep -v '\.plugins-staged' \
| xargs -r grep -Pn "(^|[^a-zA-Z0-9_])${word}([^a-zA-Z0-9_]|\$)" 2>/dev/null \
# :(glob) is required: without it git matches pathspecs with fnmatch,
# where 'shopdb/**/*.py' needs a second slash and silently skips
# top-level files such as shopdb/__init__.py and frontend/src/main.js.
HITS=$(git ls-files -z ':(glob)shopdb/**/*.py' ':(glob)shopdb/**/*.vue' ':(glob)shopdb/**/*.js' ':(glob)shopdb/**/*.ts' \
':(glob)plugins/**/*.py' ':(glob)plugins/**/*.vue' ':(glob)plugins/**/*.js' ':(glob)plugins/**/*.ts' \
':(glob)frontend/src/**/*.py' ':(glob)frontend/src/**/*.vue' ':(glob)frontend/src/**/*.js' ':(glob)frontend/src/**/*.ts' \
| grep -zv '/tests\?/' | grep -zv '\.spec\.js' | grep -zv '\.plugins-staged' \
| xargs -0 -r grep -Pn "(^|[^a-zA-Z0-9_])${word}([^a-zA-Z0-9_]|\$)" 2>/dev/null \
| grep -v '\.cs:' \
| grep -vP "(^|[^a-zA-Z0-9_])(request_obj|response_obj)" \
| drop_scratch | drop_generated || true)
@@ -209,8 +216,8 @@ done
echo "==> Checking for snake_case DB tablenames..."
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
SNAKE_TABLES=$(git ls-files '*.py' | eval "$GIT_FILES_FILTER" \
| xargs -r grep -Pn "__tablename__\s*=\s*['\"][^'\"]*_" 2>/dev/null \
SNAKE_TABLES=$(git ls-files -z '*.py' | eval "$GIT_FILES_FILTER" \
| xargs -0 -r grep -Pn "__tablename__\s*=\s*['\"][^'\"]*_" 2>/dev/null \
| drop_scratch | drop_contributing || true)
else
SNAKE_TABLES=$(grep -rPn "__tablename__\s*=\s*['\"][^'\"]*_" "${EXCLUDES[@]}" \
@@ -225,8 +232,8 @@ fi
echo "==> Checking for snake_case DB column attrs..."
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
SNAKE_COLS=$(git ls-files '*.py' | eval "$GIT_FILES_FILTER" \
| xargs -r grep -Pn "^\s+[a-z]+_[a-z_]+\s*=\s*db\.Column" 2>/dev/null \
SNAKE_COLS=$(git ls-files -z '*.py' | eval "$GIT_FILES_FILTER" \
| xargs -0 -r grep -Pn "^\s+[a-z]+_[a-z_]+\s*=\s*db\.Column" 2>/dev/null \
| drop_scratch | drop_contributing || true)
else
SNAKE_COLS=$(grep -rPn "^\s+[a-z]+_[a-z_]+\s*=\s*db\.Column" "${EXCLUDES[@]}" \
@@ -241,8 +248,8 @@ fi
echo "==> Checking for snake_case ForeignKey targets..."
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
SNAKE_FK=$(git ls-files '*.py' | eval "$GIT_FILES_FILTER" \
| xargs -r grep -Pn "ForeignKey\(['\"][^'\"]*_[^'\"]*['\"]" 2>/dev/null \
SNAKE_FK=$(git ls-files -z '*.py' | eval "$GIT_FILES_FILTER" \
| xargs -0 -r grep -Pn "ForeignKey\(['\"][^'\"]*_[^'\"]*['\"]" 2>/dev/null \
| drop_scratch || true)
else
SNAKE_FK=$(grep -rPn "ForeignKey\(['\"][^'\"]*_[^'\"]*['\"]" "${EXCLUDES[@]}" --include='*.py' . 2>/dev/null | drop_scratch || true)
@@ -256,9 +263,9 @@ fi
echo "==> Checking for snake_case API params in frontend (DB-mirrored fields)..."
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
SNAKE_FE=$(git ls-files '*.vue' '*.js' '*.ts' | eval "$GIT_FILES_FILTER" \
| grep -v 'DEVELOPMENT-STANDARDS.md' \
| xargs -r grep -Pn "params\.(machine_id|location_id|vendor_id|type_id|business_unit_id|model_id|status_id|operating_system_id|asset_id|user_id|is_active|is_shopfloor)" 2>/dev/null \
SNAKE_FE=$(git ls-files -z '*.vue' '*.js' '*.ts' | eval "$GIT_FILES_FILTER" \
| grep -zv 'DEVELOPMENT-STANDARDS.md' \
| xargs -0 -r grep -Pn "params\.(machine_id|location_id|vendor_id|type_id|business_unit_id|model_id|status_id|operating_system_id|asset_id|user_id|is_active|is_shopfloor)" 2>/dev/null \
| drop_scratch || true)
else
SNAKE_FE=$(grep -rPn "params\.(machine_id|location_id|vendor_id|type_id|business_unit_id|model_id|status_id|operating_system_id|asset_id|user_id|is_active|is_shopfloor)" "${EXCLUDES[@]}" \
@@ -278,8 +285,8 @@ fi
echo "==> Checking for cross-plugin / escaping imports in plugin frontends..."
if [ -d plugins ]; then
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
PLUGIN_FE_IMPORTS=$(git ls-files 'plugins/*/frontend/**/*.vue' 'plugins/*/frontend/**/*.js' \
| xargs -r grep -Pn "(import|from)\s+['\"]([^'\"]*\.\./\.\./|[^'\"]*/plugins/)" 2>/dev/null || true)
PLUGIN_FE_IMPORTS=$(git ls-files -z ':(glob)plugins/*/frontend/**/*.vue' ':(glob)plugins/*/frontend/**/*.js' \
| xargs -0 -r grep -Pn "(import|from)\s+['\"]([^'\"]*\.\./\.\./|[^'\"]*/plugins/)" 2>/dev/null || true)
else
PLUGIN_FE_IMPORTS=$(grep -rPn "(import|from)\s+['\"]([^'\"]*\.\./\.\./|[^'\"]*/plugins/)" \
--include='*.vue' --include='*.js' plugins/*/frontend/ 2>/dev/null || true)
@@ -305,9 +312,9 @@ fi
# passes what is specific to its label and nothing else.
echo "==> Checking that only the shared module imports the code libraries..."
if [ "$USE_GIT_LS_FILES" = "1" ] && git rev-parse --git-dir >/dev/null 2>&1; then
CODE_LIB_IMPORTS=$(git ls-files 'frontend/src/**/*.vue' 'frontend/src/**/*.js' 'plugins/**/*.vue' 'plugins/**/*.js' \
| grep -v '\.plugins-staged/' \
| xargs -r grep -n "from ['\"]\(qrcode\|jsbarcode\)['\"]" 2>/dev/null \
CODE_LIB_IMPORTS=$(git ls-files -z ':(glob)frontend/src/**/*.vue' ':(glob)frontend/src/**/*.js' ':(glob)plugins/**/*.vue' ':(glob)plugins/**/*.js' \
| grep -zv '\.plugins-staged/' \
| xargs -0 -r grep -n "from ['\"]\(qrcode\|jsbarcode\)['\"]" 2>/dev/null \
| grep -v 'frontend/src/utils/codes\.js' || true)
else
CODE_LIB_IMPORTS=$(grep -rn "from ['\"]\(qrcode\|jsbarcode\)['\"]" \

View File

@@ -1,133 +0,0 @@
param(
[Parameter(Mandatory=$false)]
[string]$Hostname = $env:COMPUTERNAME,
[Parameter(Mandatory=$false)]
[int]$ValidityYears = 2
)
# SECURITY: Validate hostname matches actual machine name to prevent cert for wrong host
$actualHostname = [System.Net.Dns]::GetHostName()
if ($Hostname -ne $actualHostname) {
Write-Warning "Hostname mismatch detected!"
Write-Warning " Requested: $Hostname"
Write-Warning " Actual machine: $actualHostname"
Write-Warning "Using actual hostname to prevent certificate mismatch."
$Hostname = $actualHostname
}
# Generate self-signed cert for this endpoint
$cert = New-SelfSignedCertificate `
-Subject "CN=$Hostname" `
-DnsName $Hostname `
-CertStoreLocation "Cert:\LocalMachine\My" `
-KeyExportPolicy NonExportable `
-KeyUsage DigitalSignature, KeyEncipherment `
-NotAfter (Get-Date).AddYears($ValidityYears) `
-HashAlgorithm SHA256
# SECURITY WARNING: Grant Users group read access to the private key so the tray
# (running as interactive user) can use it when shelling out to shopdbagent.exe --once.
# The service already works because it runs as SYSTEM.
#
# RISK: This grants ALL interactive users on the machine read access to the private
# key, which means any user with console/RDP access can extract the key and impersonate
# this endpoint. Only acceptable in environments where ALL interactive users are trusted.
#
# MITIGATION OPTIONS:
# 1. Best: Remove tray dependency on cert (tray calls service API, service makes HTTP)
# 2. Acceptable: Grant only to specific user account running the tray
# 3. Document: Only deploy in environments where all console users are trusted operators
try {
$rsaKey = [System.Security.Cryptography.X509Certificates.RSACertificateExtensions]::GetRSAPrivateKey($cert)
$keyPath = $rsaKey.Key.UniqueName
# CNG keys are in SystemKeys, not MachineKeys
$keyFile = Join-Path "$env:ProgramData\Microsoft\Crypto\SystemKeys" $keyPath
if (-not (Test-Path $keyFile)) {
$keyFile = Join-Path "$env:ProgramData\Microsoft\Crypto\Keys" $keyPath
}
if (-not (Test-Path $keyFile)) {
$keyFile = Join-Path "$env:ProgramData\Microsoft\Crypto\RSA\MachineKeys" $keyPath
}
if (Test-Path $keyFile) {
# Grant Read + Execute to BUILTIN\Users
$acl = Get-Acl $keyFile
$rule = New-Object System.Security.AccessControl.FileSystemAccessRule(
"BUILTIN\Users", "Read,ReadAndExecute", "Allow")
$acl.AddAccessRule($rule)
Set-Acl -Path $keyFile -AclObject $acl
Write-Host " Granted Users group access to private key"
} else {
Write-Host " WARNING: Could not find private key file to set permissions"
}
} catch {
Write-Host " WARNING: Could not set private key permissions: $($_.Exception.Message)"
}
Write-Host "Generated certificate:"
Write-Host " Thumbprint: $($cert.Thumbprint)"
Write-Host " Subject: $($cert.Subject)"
Write-Host " Expires: $($cert.NotAfter)"
# Compute SHA1 fingerprint to match nginx $ssl_client_fingerprint variable
# SECURITY NOTE: Nginx only provides SHA1 via $ssl_client_fingerprint variable.
# SHA1 is deprecated for cryptographic signing but remains acceptable for certificate
# fingerprinting in air-gapped environments with:
# - Short-lived certificates (2 years max)
# - Certificate revocation support (cert_revoked_at column)
# - No external CA trust (self-signed only)
# Future migration to SHA256 requires: nginx module for SHA256, or IIS deployment
# which exposes full PEM cert for server-side hashing.
$certBytes = $cert.Export([System.Security.Cryptography.X509Certificates.X509ContentType]::Cert)
$sha1 = [System.Security.Cryptography.SHA1]::Create()
$fingerprintBytes = $sha1.ComputeHash($certBytes)
$fingerprint = ($fingerprintBytes | ForEach-Object { $_.ToString("x2") }) -join ""
Write-Host ""
Write-Host "========================================="
Write-Host "CERTIFICATE GENERATED SUCCESSFULLY"
Write-Host "========================================="
Write-Host "Fingerprint (SHA1): $fingerprint"
Write-Host "Length: 40 hex chars (nginx limitation)"
Write-Host ""
Write-Host "Register this certificate in ShopDB database:"
Write-Host ""
Write-Host " UPDATE assets"
Write-Host " SET client_cert_fingerprint = '$fingerprint',"
Write-Host " cert_issued_at = CURRENT_TIMESTAMP,"
Write-Host " cert_expires_at = '$($cert.NotAfter.ToString("yyyy-MM-dd HH:mm:ss"))',"
Write-Host " cert_revoked_at = NULL,"
Write-Host " cert_revocation_reason = NULL"
Write-Host " WHERE assetnumber = '(REPLACE_WITH_ASSET_NUMBER)';"
Write-Host ""
Write-Host "To revoke this certificate later (compromised/lost endpoint):"
Write-Host ""
Write-Host " UPDATE assets"
Write-Host " SET cert_revoked_at = CURRENT_TIMESTAMP,"
Write-Host " cert_revocation_reason = 'Compromised - TICKET-123'"
Write-Host " WHERE client_cert_fingerprint = '$fingerprint';"
Write-Host ""
Write-Host "========================================="
# Update agent config
$configPath = "C:\ProgramData\ShopDB\config.json"
if (Test-Path $configPath) {
$config = Get-Content $configPath | ConvertFrom-Json
$config | Add-Member -NotePropertyName clientCertThumbprint -NotePropertyValue $cert.Thumbprint -Force
$config | ConvertTo-Json | Set-Content $configPath
Write-Host ""
Write-Host "Updated $configPath with thumbprint"
} else {
Write-Host ""
Write-Host "WARNING: $configPath not found - agent may not be configured"
}
# Output for automation
Write-Host ""
Write-Host "---MACHINE-READABLE---"
Write-Host "THUMBPRINT=$($cert.Thumbprint)"
Write-Host "FINGERPRINT=$fingerprint"
Write-Host "HOSTNAME=$Hostname"
Write-Host "EXPIRES=$($cert.NotAfter.ToString('yyyy-MM-dd HH:mm:ss'))"

View File

@@ -121,7 +121,16 @@ from .plugins import plugin_manager
# tokens are always refused by it, matching require_role, so a plugin cannot
# accidentally let a scoped token inherit role authority from its owner.
# Additive name on the surface, minor bump.
__contract_version__ = '0.32.0'
# 0.33.0 adds per-PC agent credentials to the collector contract. BasePlugin
# gains two optional hooks, resolve_agent_credential(secret) and
# agent_shared_key_refused(hostname), and shopdb.api exports
# resolve_agent_credential, agent_shared_key_refused and collector_ingest. A
# collector schema may declare "acceptsagentcredential": true; the generic
# collector then accepts X-Endpoint-Credential for it, refuses the shared key
# for an enrolled PC (401 SHARED_KEY_REFUSED), and maps a PermissionError from
# apply_collector_payload to 403 ASSET_MISMATCH. All optional, so a plugin that
# declares none of it is unaffected. Additive, minor bump.
__contract_version__ = '0.33.0'
# Product release version (see ADR-007). The product version and the
# plugin-contract version above are distinct series with independent
@@ -210,10 +219,12 @@ def create_app(config_name: str = None) -> Flask:
# Initialize plugin manager
with app.app_context():
# On first run (no users exist), log the setup token so the operator can
# create the first admin. The token gates POST /api/setup/create-admin to
# prevent a network race during the install window (API starts serving
# before the admin exists in every documented deploy path).
# On first run (no users exist), tell the operator a setup token is
# active and how to read it. The token gates POST /api/setup/create-admin
# to prevent a network race during the install window (API starts
# serving before the admin exists in every documented deploy path). The
# secret itself is never logged: logs are shipped and retained far more
# widely than the .env it came from.
from shopdb.core.models import User
try:
if User.query.count() == 0:
@@ -221,7 +232,8 @@ def create_app(config_name: str = None) -> Flask:
if token:
app.logger.warning('=' * 60)
app.logger.warning('FIRST RUN: No users exist.')
app.logger.warning('Setup token: %s', token)
app.logger.warning('A setup token is active (SETUP_TOKEN). '
'Read it with: flask seed show-setup-token')
app.logger.warning('Create admin: POST /api/setup/create-admin')
app.logger.warning(' with header: X-Setup-Token: <token>')
app.logger.warning('OR use CLI: docker exec shopdb-api flask seed admin')

View File

@@ -100,6 +100,49 @@ def collector_authorized(pluginname: str):
return _check_collector_auth(_plugin_api_key(pluginname), pluginname)
def _enabled_plugins():
from flask import current_app
manager = current_app.extensions.get('plugin_manager')
if not manager:
return []
return [plugin for name, plugin in manager.get_all_plugins().items()
if manager.registry.is_enabled(name)]
def resolve_agent_credential(secret: str) -> Optional[int]:
"""Asset id for an active per-PC agent credential, or None.
Asks every enabled plugin's resolve_agent_credential hook and returns the
first non-None answer.
"""
if not secret:
return None
for plugin in _enabled_plugins():
assetid = plugin.resolve_agent_credential(secret)
if assetid is not None:
return assetid
return None
def agent_shared_key_refused(hostname: str) -> bool:
"""True when any enabled plugin refuses the shared collector key for hostname."""
return any(plugin.agent_shared_key_refused(hostname)
for plugin in _enabled_plugins())
def collector_ingest(pluginname: str):
"""Run the generic collector ingest for a plugin on the current request.
Lets a plugin serve the same ingest under its own prefix (the endpoints
agent posts to /api/endpoints/collect) without importing core collector
internals. Auth, validation, audit and response are exactly those of
POST /api/collector/<pluginname>.
"""
from shopdb.core.api.collector import generic_collect
return generic_collect(pluginname)
def application_package_location():
"""Where application installers live, and the URL prefix that serves them.
@@ -421,6 +464,9 @@ __all__ = [
'list_service_tokens',
'revoke_service_tokens',
'collector_authorized',
'resolve_agent_credential',
'agent_shared_key_refused',
'collector_ingest',
'application_package_location',
'script_body_location',
'SupportTeam',

View File

@@ -205,8 +205,8 @@ def show_setup_token():
"""Display the setup token for first-admin creation.
The token gates POST /api/setup/create-admin so a network race cannot claim
the instance during the install window. Printed at first startup; retrieve
it here if the logs rotated or you lost the terminal.
the instance during the install window. Startup only logs that a token is
active (never the secret itself); this command is where it is read.
"""
import os
from flask import current_app

View File

@@ -60,8 +60,18 @@ def _requires_policy_ack(app, data):
POLICY_ACK_MESSAGE = (
'Marking this application as required on all PCs creates a configuration '
'policy that will install it on every non-server endpoint it matches. '
'Confirm to create the policy.'
'policy that installs it on every non-server endpoint it matches, once that '
'policy is added to an active business policy. It is not added to one '
'automatically. Confirm to create the policy.'
)
# Returned alongside policyresult='created'. The generated policy is not
# attached to any BusinessPolicy, and the resolver only applies policies reached
# through one, so saying "this will install everywhere" would be untrue.
POLICY_CREATED_NOTE = (
'A configuration policy was created for this application, but it is not '
'attached to any business policy and installs nothing until an admin adds '
'it to one.'
)
# Deleting an application is irreversible and takes its versions and uploaded
@@ -120,6 +130,24 @@ def _validate_monitorable(data, existing=None):
return None
def _policy_payload(payload, policyresult):
"""Attach the policy sync outcome (and the not-attached note) to a response."""
if policyresult in ('created', 'removed', 'kept'):
payload['policyresult'] = policyresult
if policyresult == 'created':
payload['policynote'] = POLICY_CREATED_NOTE
return payload
def _deactivate_policies_for_app(app_id):
"""Deactivate remaining policies that reference a deleted appid, if available."""
try:
from plugins.computers.services.apppolicy import deactivate_policies_for_app
except ImportError:
return []
return deactivate_policies_for_app(app_id)
def _sync_required_app_policy(app):
"""Sync the auto-generated "required on all PCs" policy, if available.
@@ -397,9 +425,7 @@ def create_application():
db.session.commit()
payload = app.to_dict()
if policyresult in ('created', 'removed', 'kept'):
payload['policyresult'] = policyresult
payload = _policy_payload(app.to_dict(), policyresult)
return success_response(payload, message='Application created', http_code=201)
@@ -439,6 +465,20 @@ def update_application(app_id: int):
'ismonitorable', 'monitorlaunchpath', 'monitorprocessname'
]
# Only touch policy when the checkbox itself moved. Syncing on every save
# would let an unrelated edit resurrect a policy an admin had deleted.
#
# Unticking is synced BEFORE any field is applied, so a PUT that renames and
# unticks in one go withdraws the policy that belongs to the app as it was,
# not one looked up under the new name.
policyresult = None
unticking = ('isrequired' in data and bool(app.isrequired)
and not data['isrequired'])
if unticking:
app.isrequired = False
policyresult = _sync_required_app_policy(app)
app.isrequired = True
changes = {}
for key in fields:
if key in data:
@@ -452,18 +492,16 @@ def update_application(app_id: int):
AuditLog.log('updated', 'Application', entityid=app.appid,
entityname=app.appname, changes=changes)
# Only touch policy when the checkbox itself moved. Syncing on every save
# would let an unrelated edit resurrect a policy an admin had deleted.
policyresult = None
if 'isrequired' in changes:
if 'isrequired' in changes and not unticking:
policyresult = _sync_required_app_policy(app)
elif 'appname' in changes and app.isrequired:
# A rename of a required app keeps its generated policy's name current.
_sync_required_app_policy(app)
apply_import_timestamps(app, data)
db.session.commit()
payload = app.to_dict()
if policyresult in ('created', 'removed', 'kept'):
payload['policyresult'] = policyresult
payload = _policy_payload(app.to_dict(), policyresult)
return success_response(payload, message='Application updated')
@@ -561,7 +599,13 @@ def delete_application(app_id: int):
# application would keep installing something that no longer exists.
appname = app.appname
app.isrequired = False
_sync_required_app_policy(app)
policyresult = _sync_required_app_policy(app)
# 'kept' means a customised generated policy survived the sync, and other
# hand-authored policies may name this appid too. Every one of them would
# keep trying to install an application that no longer exists, so they are
# deactivated (not deleted - they are someone's work) and reported.
deactivated = _deactivate_policies_for_app(app_id)
_clear_application_references(app_id)
removed = _remove_upload_files(app_id)
@@ -570,12 +614,26 @@ def delete_application(app_id: int):
# The audit row is the only record left that this application existed, so
# it names the files that went with it.
auditchanges = {}
if removed:
auditchanges['removedfiles'] = removed
if deactivated:
auditchanges['deactivatedpolicies'] = deactivated
AuditLog.log('deleted', 'Application', entityid=app_id, entityname=appname,
changes={'removedfiles': removed} if removed else None)
changes=auditchanges or None)
db.session.commit()
return success_response(message='Application deleted')
payload = {}
if policyresult in ('removed', 'kept'):
payload['policyresult'] = policyresult
if deactivated:
payload['deactivatedpolicies'] = deactivated
message = 'Application deleted'
if deactivated:
message += ('; deactivated {} policy(ies) that referenced it: {}'
.format(len(deactivated), ', '.join(deactivated)))
return success_response(payload or None, message=message)
REVERT_CONFIRM_MESSAGE = (

View File

@@ -13,7 +13,6 @@ from flask import Blueprint, request, current_app, g
from shopdb.extensions import db
from shopdb.core.models import Asset, Application
from shopdb.core.models.apitoken import ApiToken, TOKEN_SECRET_PREFIX
from shopdb.core.auth.client_cert import authenticate_by_client_cert
from shopdb.utils.responses import success_response, error_response, ErrorCodes
@@ -24,6 +23,10 @@ collector_bp = Blueprint('collector', __name__)
# identity - no require_permission on these routes).
COLLECTOR_SCOPE = 'collector.ingest'
# Per-PC agent credential (contract 0.33.0). Accepted only by collector schemas
# declaring acceptsagentcredential.
AGENT_CREDENTIAL_HEADER = 'X-Endpoint-Credential'
def _computer_models():
"""Lazily import the computers plugin models.
@@ -242,21 +245,28 @@ def generic_collect(pluginname):
plugin, schema = plugins[pluginname]
# Dual-auth: try client certificate first (per-endpoint mTLS), fall back to
# per-plugin env key or collector-scoped managed token. Header only
# (querystring fallback dropped, see require_api_key).
from shopdb.core.auth.client_cert import authenticate_by_client_cert
cert_asset = authenticate_by_client_cert()
if cert_asset:
g.cert_authenticated_asset_id = cert_asset.assetid
current_app.logger.debug(
'[collector] Authenticated via client cert for asset %d',
cert_asset.assetid)
# A schema that accepts agent credentials takes X-Endpoint-Credential and
# derives the calling asset from it. Otherwise: per-plugin env key or a
# collector-scoped managed token, header only (see require_api_key).
from shopdb.api import resolve_agent_credential, agent_shared_key_refused
acceptscredential = bool(schema.get('acceptsagentcredential'))
g.agentauth = None
g.agentassetid = None
secret = (request.headers.get(AGENT_CREDENTIAL_HEADER) or '').strip()
if secret and acceptscredential:
assetid = resolve_agent_credential(secret)
if assetid is None:
return error_response('CREDENTIAL_INVALID',
'Endpoint credential is unknown or revoked; '
'enroll again', http_code=401)
g.agentauth = 'credential'
g.agentassetid = assetid
else:
# No cert or cert invalid: fall back to API key
error = _check_collector_auth(_plugin_api_key(pluginname), pluginname)
if error is not None:
return error
g.agentauth = 'sharedkey'
payload = request.get_json(silent=True)
if not payload or not isinstance(payload, dict):
@@ -268,6 +278,12 @@ def generic_collect(pluginname):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'{identityfield} is required')
if (acceptscredential and g.agentauth == 'sharedkey'
and agent_shared_key_refused(identityvalue)):
return error_response('SHARED_KEY_REFUSED',
'The shared key is not accepted for this '
'endpoint; use its credential', http_code=401)
try:
outcome = plugin.apply_collector_payload(payload)
except NotImplementedError:
@@ -275,6 +291,12 @@ def generic_collect(pluginname):
ErrorCodes.INTERNAL_ERROR,
f'Plugin {pluginname} does not implement apply_collector_payload',
http_code=500)
except PermissionError as exc:
# The payload names a different asset than the agent credential.
db.session.rollback()
return error_response('ASSET_MISMATCH', str(exc) or
'The payload names a different endpoint than '
'the credential', http_code=403)
except ValueError as exc:
# ValueError is the plugin's controlled validation signal; its message
# is safe to return to the caller.

View File

@@ -61,10 +61,18 @@ def get_plugin_settings_routes():
# Only include routes from enabled plugins
if not pm.registry.is_enabled(plugin_name):
continue
if hasattr(plugin, 'get_settings_routes'):
if not hasattr(plugin, 'get_settings_routes'):
continue
# One broken plugin must not take every settings hub down with it.
try:
plugin_routes = plugin.get_settings_routes()
if plugin_routes:
routes.extend(plugin_routes)
except Exception:
current_app.logger.exception(
'get_settings_routes failed for plugin %s; skipping it',
plugin_name)
continue
if plugin_routes:
routes.extend(plugin_routes)
return success_response({'routes': routes})

View File

@@ -1,213 +0,0 @@
"""Client certificate authentication helpers.
Extracts client certificate fingerprints from various deployment types:
- IIS 2019: SSL_CLIENT_CERT environ (PEM format)
- Nginx: X-Client-Cert-Fingerprint header (SHA1 hex, colon-separated)
- IIS with ARR: X-ARR-ClientCert header (base64 DER)
SECURITY: Nginx $ssl_client_fingerprint outputs SHA1 (40 hex chars), not SHA256.
This is a known Nginx limitation. SHA1 is deprecated for signing but remains
acceptable for fingerprinting self-signed certificates in air-gapped environments
with short-lived certs and revocation support.
"""
import hashlib
import base64
import hmac
import re
from flask import request, current_app
def get_client_cert_fingerprint():
"""Extract client certificate fingerprint from request.
Returns:
str: 40-character lowercase hex SHA1 fingerprint, or None if no cert
Supports multiple deployment types:
- Nginx: reads X-Client-Cert-Fingerprint header (SHA1, colon-separated)
- IIS direct: reads SSL_CLIENT_CERT environ (PEM format, computes SHA256)
- IIS with ARR: reads X-ARR-ClientCert header (base64 DER, computes SHA256)
SECURITY: Nginx returns SHA1 (40 hex chars). IIS paths compute SHA256 (64 hex
chars). The database client_cert_fingerprint column holds 64 chars to support
future SHA256 migration but currently stores SHA1 from Nginx deployments.
"""
# Option 1: Nginx passes pre-computed SHA1 fingerprint
# Format: "AB:CD:EF:..." (colon-separated uppercase hex)
nginx_fp = request.headers.get('X-Client-Cert-Fingerprint')
if nginx_fp:
# Normalize: remove colons, lowercase
normalized = nginx_fp.replace(':', '').lower()
# Validate format: must be 40 hex chars (SHA1) or 64 hex chars (SHA256)
if len(normalized) == 40:
# SHA1 from nginx $ssl_client_fingerprint
if not re.match(r'^[0-9a-f]{40}$', normalized):
current_app.logger.warning(
f'[client_cert] Invalid SHA1 fingerprint format: {normalized[:16]}...')
return None
return normalized
elif len(normalized) == 64:
# SHA256 from custom nginx config or future migration
if not re.match(r'^[0-9a-f]{64}$', normalized):
current_app.logger.warning(
f'[client_cert] Invalid SHA256 fingerprint format: {normalized[:16]}...')
return None
return normalized
else:
current_app.logger.warning(
f'[client_cert] Invalid fingerprint length: {len(normalized)} '
f'(expected 40 for SHA1 or 64 for SHA256)')
return None
# Option 2: IIS passes full cert in PEM format via WSGI environ
iis_cert_pem = request.environ.get('SSL_CLIENT_CERT')
if iis_cert_pem:
try:
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
# Parse PEM certificate
if isinstance(iis_cert_pem, str):
iis_cert_pem = iis_cert_pem.encode('utf-8')
cert = x509.load_pem_x509_certificate(iis_cert_pem, default_backend())
# Convert to DER and hash
cert_der = cert.public_bytes(serialization.Encoding.DER)
fingerprint = hashlib.sha256(cert_der).hexdigest()
current_app.logger.debug(
f'[client_cert] Extracted from IIS SSL_CLIENT_CERT: {fingerprint[:16]}...')
return fingerprint
except Exception as e:
current_app.logger.warning(f'[client_cert] Failed to parse IIS cert: {e}')
return None
# Option 3: IIS with ARR passes base64-encoded DER
arr_cert_b64 = request.headers.get('X-ARR-ClientCert')
if arr_cert_b64:
try:
# Decode base64 to get DER bytes
cert_der = base64.b64decode(arr_cert_b64)
fingerprint = hashlib.sha256(cert_der).hexdigest()
current_app.logger.debug(
f'[client_cert] Extracted from ARR header: {fingerprint[:16]}...')
return fingerprint
except Exception as e:
current_app.logger.warning(f'[client_cert] Failed to parse ARR cert: {e}')
return None
# No client certificate present
return None
def authenticate_by_client_cert():
"""Authenticate request by client certificate.
Returns:
Asset: The authenticated asset, or None if cert auth fails
Checks:
1. Client cert present in request?
2. Fingerprint matches an asset in database?
3. Certificate not revoked?
4. Certificate not expired?
Does NOT raise exceptions - returns None on any failure.
Caller decides whether to fall back to other auth methods.
AUDIT: Logs all authentication attempts (success and failure) to AuditLog.
"""
from shopdb.api import Asset, AuditLog
from datetime import datetime, timezone
fingerprint = get_client_cert_fingerprint()
if not fingerprint:
return None
# Look up asset by cert fingerprint
asset = Asset.query.filter_by(client_cert_fingerprint=fingerprint).first()
if not asset:
current_app.logger.warning(
f'[client_cert] Unknown fingerprint: {fingerprint[:16]}... '
f'(not associated with any asset)')
# Audit: Failed authentication attempt with unknown fingerprint
try:
AuditLog.log(
action='auth.cert.unknown',
userid=None,
details=f'Unknown fingerprint: {fingerprint[:16]}...')
except Exception:
pass # Don't fail authentication on audit failure
return None
# Constant-time comparison to prevent timing attacks. Database lookup already
# happened (timing leak there is unavoidable with indexed lookups), but this
# prevents attackers from using response timing to confirm partial fingerprint
# matches during the final validation.
if not hmac.compare_digest(asset.client_cert_fingerprint, fingerprint):
current_app.logger.warning(
f'[client_cert] Fingerprint mismatch for asset {asset.assetid} '
f'(database vs request)')
return None
# Check certificate revocation (cert_revoked_at is set when admin revokes)
# Fail closed: if column missing (should never happen after migration 0022),
# treat as database error and reject authentication
if asset.cert_revoked_at:
current_app.logger.warning(
f'[client_cert] Revoked certificate for asset {asset.assetid} '
f'(revoked {asset.cert_revoked_at}: {asset.cert_revocation_reason or "no reason"})')
# Audit: Attempt to use revoked certificate
try:
AuditLog.log(
action='auth.cert.revoked',
userid=None,
entityid=asset.assetid,
entityname=asset.assetnumber,
details=f'Revoked at {asset.cert_revoked_at}: '
f'{asset.cert_revocation_reason or "no reason"}')
except Exception:
pass # Don't fail authentication on audit failure
return None
# Check certificate expiration
if asset.cert_expires_at:
now = datetime.now(timezone.utc).replace(tzinfo=None)
if asset.cert_expires_at < now:
current_app.logger.warning(
f'[client_cert] Expired certificate for asset {asset.assetid} '
f'(expired {asset.cert_expires_at})')
# Audit: Attempt to use expired certificate
try:
AuditLog.log(
action='auth.cert.expired',
userid=None,
entityid=asset.assetid,
entityname=asset.assetnumber,
details=f'Expired {asset.cert_expires_at}')
except Exception:
pass
return None
current_app.logger.info(
f'[client_cert] Authenticated asset {asset.assetid} '
f'(number={asset.assetnumber}) via certificate {fingerprint[:16]}...')
# Audit: Successful certificate authentication
try:
AuditLog.log(
action='auth.cert.success',
userid=None,
entityid=asset.assetid,
entityname=asset.assetnumber,
details=f'Cert fingerprint: {fingerprint[:16]}...')
except Exception:
pass
return asset

View File

@@ -139,34 +139,6 @@ class Asset(BaseModel, SoftDeleteMixin, AuditMixin):
# Notes
notes = db.Column(db.Text, nullable=True)
# Client certificate authentication (endpoints plugin)
client_cert_fingerprint = db.Column(
db.String(64),
nullable=True,
index=True,
comment='SHA1 or SHA256 fingerprint of client certificate for mTLS auth'
)
cert_issued_at = db.Column(
db.DateTime,
nullable=True,
comment='When the client certificate was issued'
)
cert_expires_at = db.Column(
db.DateTime,
nullable=True,
comment='When the certificate expires'
)
cert_revoked_at = db.Column(
db.DateTime,
nullable=True,
comment='When the certificate was revoked (null = not revoked)'
)
cert_revocation_reason = db.Column(
db.String(255),
nullable=True,
comment='Why the certificate was revoked (admin note)'
)
# Relationships
assettype = db.relationship('AssetType', backref='assets')
status = db.relationship('AssetStatus', backref='assets')

View File

@@ -58,7 +58,9 @@ PLUGIN_TABLE_OWNERS: dict[str, Iterable[str]] = {
'endpointapplications', 'endpointmonitors',
'operatorreports', 'endpointhealthstates',
'endpointhealthtransitions', 'endpointmonitorimpacts',
'endpointlocks', 'endpointlogcollections'),
'endpointlocks', 'endpointlogcollections',
'endpointcredentials', 'endpointenrollrequests',
'endpointmigrationwaves'),
'geenforce': ('manifestscopes', 'manifestentries', 'manifestentrypctypes',
'manifestentryhostnames', 'manifestentrymachinenumbers',
'manifestinusechecks', 'manifestinusecheckprocesses',

View File

@@ -236,6 +236,27 @@ class BasePlugin(ABC):
f"implement apply_collector_payload"
)
def resolve_agent_credential(self, secret: str) -> Optional[int]:
"""Asset id for an active per-PC agent credential, else None.
Optional (contract 0.33.0). A plugin that issues endpoint credentials
answers here so the generic collector can accept them on any schema
declaring 'acceptsagentcredential'. Called via
shopdb.api.resolve_agent_credential, which asks enabled plugins and
takes the first non-None answer.
"""
return None
def agent_shared_key_refused(self, hostname: str) -> bool:
"""True when the shared collector key may not report for this hostname.
Optional (contract 0.33.0). True when the shared key is switched off or
the hostname's asset holds an active per-PC credential. Called via
shopdb.api.agent_shared_key_refused, which is True if any enabled
plugin says so.
"""
return False
def on_install(self, app: Flask) -> None:
"""Called when plugin is installed via CLI."""
pass

View File

@@ -1,182 +0,0 @@
# Test certificate renewal process
# This script validates end-to-end certificate renewal
$ErrorActionPreference = "Stop"
Write-Host "=== Certificate Renewal Test ===" -ForegroundColor Cyan
Write-Host ""
# 1. Check current certificate
Write-Host "Step 1: Check current certificate" -ForegroundColor Yellow
$assetId = 2528
$query = @"
SELECT
assetid,
assetnumber,
client_cert_fingerprint,
cert_issued_at,
cert_expires_at,
DATEDIFF(cert_expires_at, NOW()) as days_remaining
FROM assets
WHERE assetid = $assetId
"@
Write-Host "Querying database for current cert status..."
$result = docker-compose exec -T db mysql -u root -proot-dev-password-not-for-production shopdb_flask -sN -e $query 2>$null
if ($result) {
$fields = $result -split "`t"
Write-Host " Asset: $($fields[1])" -ForegroundColor Green
Write-Host " Current Fingerprint: $($fields[2])" -ForegroundColor Green
Write-Host " Issued: $($fields[3])" -ForegroundColor Green
Write-Host " Expires: $($fields[4])" -ForegroundColor Green
Write-Host " Days Remaining: $($fields[5])" -ForegroundColor Green
$oldFingerprint = $fields[2]
$daysRemaining = [int]$fields[5]
} else {
Write-Host " ERROR: Could not query database" -ForegroundColor Red
exit 1
}
Write-Host ""
# 2. Test renewal endpoint
Write-Host "Step 2: Test renewal endpoint (API key fallback)" -ForegroundColor Yellow
$apiKey = (Get-Content .env | Select-String "COLLECTOR_API_KEY_ENDPOINTS" | ForEach-Object { $_.ToString().Split('=')[1] })
if (-not $apiKey) {
Write-Host " ERROR: Could not get API key from .env" -ForegroundColor Red
exit 1
}
$body = @{
hostname = "DESKTOP-HF1K9M4"
} | ConvertTo-Json
Write-Host " Calling POST https://localhost:5001/api/endpoints/certificate/renew"
Write-Host " (This may take a few seconds...)"
try {
$response = Invoke-RestMethod `
-Uri "https://localhost:5001/api/endpoints/certificate/renew" `
-Method POST `
-Headers @{
"X-API-Key" = $apiKey
"Content-Type" = "application/json"
} `
-Body $body `
-SkipCertificateCheck `
-ErrorAction Stop
if ($response.status -eq "success") {
Write-Host " ✓ Renewal succeeded!" -ForegroundColor Green
$newThumbprint = $response.data.thumbprint
$newExpires = $response.data.expires_at
$pfxSize = [System.Convert]::FromBase64String($response.data.pfx).Length
Write-Host " New Thumbprint: $newThumbprint" -ForegroundColor Green
Write-Host " New Expires: $newExpires" -ForegroundColor Green
Write-Host " PFX Size: $pfxSize bytes" -ForegroundColor Green
Write-Host " Password Length: $($response.data.password.Length) chars" -ForegroundColor Green
} else {
Write-Host " × Renewal failed: $($response.data.error.message)" -ForegroundColor Red
if ($response.data.error.code -eq "VALIDATION_ERROR" -and
$response.data.error.message -like "*24 hours*") {
Write-Host ""
Write-Host " NOTE: Certificate was renewed < 24 hours ago (rate limited)" -ForegroundColor Yellow
Write-Host " This is expected behavior and the security fix is working correctly." -ForegroundColor Yellow
Write-Host ""
Write-Host " To test renewal:" -ForegroundColor Cyan
Write-Host " 1. Wait 24 hours, OR" -ForegroundColor Cyan
Write-Host " 2. Manually update cert_issued_at in database:" -ForegroundColor Cyan
Write-Host " UPDATE assets SET cert_issued_at = DATE_SUB(NOW(), INTERVAL 25 HOUR) WHERE assetid = $assetId;" -ForegroundColor Gray
exit 0
}
}
} catch {
Write-Host " × Request failed: $($_.Exception.Message)" -ForegroundColor Red
if ($_.ErrorDetails.Message) {
$errorData = $_.ErrorDetails.Message | ConvertFrom-Json
Write-Host " Error: $($errorData.data.error.message)" -ForegroundColor Red
}
exit 1
}
Write-Host ""
# 3. Verify database was updated
Write-Host "Step 3: Verify database was updated" -ForegroundColor Yellow
$newResult = docker-compose exec -T db mysql -u root -proot-dev-password-not-for-production shopdb_flask -sN -e $query 2>$null
if ($newResult) {
$newFields = $newResult -split "`t"
$dbFingerprint = $newFields[2]
$dbExpires = $newFields[4]
if ($dbFingerprint -eq $oldFingerprint) {
Write-Host " × Database not updated (fingerprint unchanged)" -ForegroundColor Red
exit 1
} else {
Write-Host " ✓ Database updated successfully" -ForegroundColor Green
Write-Host " Old Fingerprint: $oldFingerprint" -ForegroundColor Gray
Write-Host " New Fingerprint: $dbFingerprint" -ForegroundColor Green
Write-Host " New Expires: $dbExpires" -ForegroundColor Green
}
}
Write-Host ""
# 4. Test hostname validation (security fix #4)
Write-Host "Step 4: Test hostname validation (Security Fix #4)" -ForegroundColor Yellow
$testCases = @(
@{ Hostname = ""; Description = "Empty hostname"; ExpectedStatus = 400 }
@{ Hostname = "x" * 65; Description = "Too long (65 chars)"; ExpectedStatus = 400 }
@{ Hostname = "test;DROP TABLE"; Description = "SQL injection"; ExpectedStatus = 400 }
@{ Hostname = "../../etc/passwd"; Description = "Path traversal"; ExpectedStatus = 400 }
)
$passed = 0
foreach ($test in $testCases) {
$testBody = @{ hostname = $test.Hostname } | ConvertTo-Json
try {
$testResponse = Invoke-RestMethod `
-Uri "https://localhost:5001/api/endpoints/certificate/renew" `
-Method POST `
-Headers @{
"X-API-Key" = $apiKey
"Content-Type" = "application/json"
} `
-Body $testBody `
-SkipCertificateCheck `
-ErrorAction Stop
Write-Host " × $($test.Description): Got 200, expected $($test.ExpectedStatus)" -ForegroundColor Red
} catch {
$statusCode = $_.Exception.Response.StatusCode.value__
if ($statusCode -eq $test.ExpectedStatus) {
Write-Host " ✓ $($test.Description): Rejected correctly ($statusCode)" -ForegroundColor Green
$passed++
} else {
Write-Host " × $($test.Description): Got $statusCode, expected $($test.ExpectedStatus)" -ForegroundColor Red
}
}
}
Write-Host " Validation tests: $passed/$($testCases.Count) passed" -ForegroundColor $(if ($passed -eq $testCases.Count) { "Green" } else { "Yellow" })
Write-Host ""
# Summary
Write-Host "=== Test Summary ===" -ForegroundColor Cyan
Write-Host "✓ Certificate renewal endpoint operational" -ForegroundColor Green
Write-Host "✓ Database updates working" -ForegroundColor Green
Write-Host "✓ Hostname validation enforced" -ForegroundColor Green
Write-Host "✓ Security fixes validated" -ForegroundColor Green
Write-Host ""
Write-Host "NOTE: Agent must be rebuilt and deployed to test:" -ForegroundColor Yellow
Write-Host " - Periodic expiry monitoring (24-hour checks)" -ForegroundColor Yellow
Write-Host " - Graceful service restart after renewal" -ForegroundColor Yellow
Write-Host " - Server certificate validation" -ForegroundColor Yellow

View File

@@ -1,212 +0,0 @@
#!/usr/bin/env python3
"""Test certificate renewal endpoint.
Tests all security fixes:
- Hostname validation
- Rate limiting
- API key fallback authentication
- Certificate generation
- Database updates
"""
import requests
import json
import base64
from datetime import datetime
# Test configuration
BASE_URL = "https://localhost:5001"
API_KEY = "shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43"
HOSTNAME = "DESKTOP-HF1K9M4"
# Disable SSL warnings for self-signed cert
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
def test_renewal_with_api_key():
"""Test renewal using API key + hostname fallback (for expired certs)."""
print("\n=== Test 1: Renewal with API Key + Hostname ===")
url = f"{BASE_URL}/api/endpoints/certificate/renew"
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
payload = {"hostname": HOSTNAME}
print(f"POST {url}")
print(f"Payload: {payload}")
response = requests.post(url, headers=headers, json=payload, verify=False)
print(f"Status: {response.status_code}")
print(f"Response: {response.text[:500]}")
if response.status_code == 200:
data = response.json()["data"]
print(f"\n✅ SUCCESS!")
print(f" Thumbprint: {data['thumbprint'][:16]}...")
print(f" Expires: {data['expires_at']}")
print(f" PFX size: {len(base64.b64decode(data['pfx']))} bytes")
print(f" Password length: {len(data['password'])} chars")
return True
elif response.status_code == 429:
print(f"\n⚠️ RATE LIMITED (expected if cert renewed < 24h ago)")
return True
else:
print(f"\n❌ FAILED")
return False
def test_hostname_validation():
"""Test hostname validation (Fix #4)."""
print("\n=== Test 2: Hostname Validation ===")
url = f"{BASE_URL}/api/endpoints/certificate/renew"
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
test_cases = [
("", "empty hostname", 400),
("x" * 65, "hostname too long", 400),
("test;DROP TABLE", "SQL injection attempt", 400),
("../../etc/passwd", "path traversal", 400),
("test\x00null", "null byte", 400),
("VALID-HOST", "valid hostname", 200 or 429 or 404), # 404 if hostname unknown
]
passed = 0
for hostname, description, expected in test_cases:
payload = {"hostname": hostname}
response = requests.post(url, headers=headers, json=payload, verify=False)
if isinstance(expected, tuple):
success = response.status_code in expected
else:
success = response.status_code == expected
status = "✅" if success else "❌"
print(f" {status} {description}: {response.status_code}")
if success:
passed += 1
print(f"\nValidation Tests: {passed}/{len(test_cases)} passed")
return passed == len(test_cases)
def test_rate_limiting():
"""Test rate limiting (Fix #5)."""
print("\n=== Test 3: Rate Limiting ===")
url = f"{BASE_URL}/api/endpoints/certificate/renew"
headers = {
"X-API-Key": API_KEY,
"Content-Type": "application/json"
}
payload = {"hostname": HOSTNAME}
print("Making first renewal request...")
response1 = requests.post(url, headers=headers, json=payload, verify=False)
print(f" First request: {response1.status_code}")
if response1.status_code == 200:
print("\nMaking immediate second request (should be rate limited)...")
response2 = requests.post(url, headers=headers, json=payload, verify=False)
print(f" Second request: {response2.status_code}")
if response2.status_code == 429:
print("✅ Rate limiting works! Certificate can only be renewed once per 24 hours.")
return True
else:
print("❌ Rate limiting failed - second request should return 429")
return False
elif response1.status_code == 429:
print("✅ Already rate limited (cert renewed < 24h ago)")
return True
else:
print(f"⚠️ Cannot test rate limiting - first request returned {response1.status_code}")
return False
def test_certificate_info():
"""Check current certificate status in database."""
print("\n=== Test 4: Current Certificate Status ===")
# Query asset info via API (requires JWT auth, so this is a basic check)
url = f"{BASE_URL}/api/endpoints/test/client-cert"
try:
response = requests.get(url, verify=False)
if response.status_code == 200:
data = response.json()
print(f" Fingerprint in DB: {data.get('fingerprint', 'N/A')}")
print(f" Issued: {data.get('issued_at', 'N/A')}")
print(f" Expires: {data.get('expires_at', 'N/A')}")
print(f" Revoked: {data.get('revoked_at', 'N/A') or 'No'}")
return True
except Exception as e:
print(f" ℹ️ Cannot query cert status (requires auth): {e}")
return True
def test_audit_logging():
"""Verify audit logs are created (Fix #8)."""
print("\n=== Test 5: Audit Logging ===")
print(" ℹ️ Check database auditlogs table for:")
print(" - action='endpoints.cert.renewed' on success")
print(" - action='endpoints.cert.renewal_failed' on cert generation failure")
print(" - action='endpoints.cert.db_update_failed' on database failure")
print(" Manual verification required (needs DB access)")
return True
def main():
"""Run all tests."""
print("=" * 60)
print("Certificate Renewal Security Test Suite")
print("=" * 60)
print(f"\nTarget: {BASE_URL}")
print(f"Hostname: {HOSTNAME}")
print(f"Testing fixes from Fable security review")
# Check if API key is configured
if API_KEY == "your_collector_api_key_here":
print("\n❌ ERROR: Update API_KEY in script with actual collector key")
print(" Get it from: HKLM\\SOFTWARE\\GE\\ShopDB\\CollectorKey")
print(" Or from: docker-compose.yml COLLECTOR_API_KEY_ENDPOINTS")
return
results = []
# Run tests
results.append(("Renewal with API Key", test_renewal_with_api_key()))
results.append(("Hostname Validation", test_hostname_validation()))
results.append(("Rate Limiting", test_rate_limiting()))
results.append(("Certificate Info", test_certificate_info()))
results.append(("Audit Logging", test_audit_logging()))
# Summary
print("\n" + "=" * 60)
print("TEST SUMMARY")
print("=" * 60)
for name, passed in results:
status = "✅ PASS" if passed else "❌ FAIL"
print(f"{status:12} {name}")
passed_count = sum(1 for _, p in results if p)
total_count = len(results)
print(f"\nTotal: {passed_count}/{total_count} tests passed")
if passed_count == total_count:
print("\n✅ All security fixes validated!")
else:
print(f"\n⚠️ {total_count - passed_count} test(s) failed")
if __name__ == "__main__":
main()

View File

@@ -1,45 +0,0 @@
#!/bin/bash
echo "=== Certificate Renewal Security Tests ==="
echo ""
API_KEY="shopdb_pat_b4a3fe2a01abc6bc1cb685280e65a8c35d52ec43"
BASE_URL="https://localhost:5001"
echo "Test 1: Hostname Validation - Empty hostname"
curl -k -X POST "$BASE_URL/api/endpoints/certificate/renew" \
-H "X-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"hostname":""}' \
-s -w "\nStatus: %{http_code}\n\n" | grep -E "error|Status"
echo "Test 2: Hostname Validation - SQL injection attempt"
curl -k -X POST "$BASE_URL/api/endpoints/certificate/renew" \
-H "X-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"hostname":"test;DROP TABLE"}' \
-s -w "\nStatus: %{http_code}\n\n" | grep -E "error|Status"
echo "Test 3: Hostname Validation - Path traversal"
curl -k -X POST "$BASE_URL/api/endpoints/certificate/renew" \
-H "X-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"hostname":"../../etc/passwd"}' \
-s -w "\nStatus: %{http_code}\n\n" | grep -E "error|Status"
echo "Test 4: Hostname Validation - Too long (65 chars)"
curl -k -X POST "$BASE_URL/api/endpoints/certificate/renew" \
-H "X-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"hostname":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}' \
-s -w "\nStatus: %{http_code}\n\n" | grep -E "error|Status"
echo "Test 5: Valid hostname (should work or rate limit)"
curl -k -X POST "$BASE_URL/api/endpoints/certificate/renew" \
-H "X-API-Key: $API_KEY" \
-H "Content-Type: application/json" \
-d '{"hostname":"DESKTOP-HF1K9M4"}' \
-s -w "\nStatus: %{http_code}\n" | head -20
echo ""
echo "=== Check Agent Cert Authentication ==="
docker-compose logs --tail=500 api 2>&1 | grep "Authenticated asset 2528" | tail -3

View File

@@ -0,0 +1,117 @@
"""scripts/backfill_app_fingerprints.py against a real (SQLite) session.
The unique index on (assetid, appfingerprint) turns two identical inventory rows
on one asset into an IntegrityError mid-backfill. The script must keep the
lowest id, delete the rest and report how many, and a failing batch must be
rolled back rather than left half-applied.
"""
import importlib.util
from pathlib import Path
import pytest
SCRIPT = Path(__file__).resolve().parent.parent / 'scripts' / 'backfill_app_fingerprints.py'
@pytest.fixture
def backfill(app):
spec = importlib.util.spec_from_file_location('backfillfingerprints', SCRIPT)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _asset(db, number):
from shopdb.core.models import Asset, AssetType
atype = AssetType.query.filter_by(assettype='computer').first()
if not atype:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
asset = Asset(assetnumber=number, assettypeid=atype.assettypeid,
isactive=True)
db.session.add(asset)
db.session.commit()
return asset.assetid
def _row(db, assetid, name, version='1.0', publisher='Vendor',
fingerprint=None):
from plugins.endpoints.models import EndpointApplication
row = EndpointApplication(assetid=assetid, displayname=name,
displayversion=version, publisher=publisher,
**{'appfingerprint': fingerprint})
db.session.add(row)
db.session.commit()
return row.endpointapplicationid
def test_self_check_uses_fixed_expected_values(backfill, monkeypatch):
assert backfill.verify_fingerprint_algorithm() is True
monkeypatch.setattr(backfill, 'compute_fingerprint',
lambda name, version, publisher: '0' * 64)
with pytest.raises(ValueError):
backfill.verify_fingerprint_algorithm()
def test_duplicates_keep_the_lowest_id(backfill, db):
from plugins.endpoints.models import EndpointApplication
first = _asset(db, 'FP01')
second = _asset(db, 'FP02')
keep = _row(db, first, 'Chrome')
_row(db, first, 'Chrome')
_row(db, first, 'Chrome')
other = _row(db, first, 'Office')
elsewhere = _row(db, second, 'Chrome')
summary = backfill.backfill_fingerprints(batch_size=2)
assert summary['duplicatesremoved'] == 2
assert summary['failed'] == 0
rows = EndpointApplication.query.order_by(
EndpointApplication.endpointapplicationid).all()
assert [r.endpointapplicationid for r in rows] == [keep, other, elsewhere]
assert all(r.appfingerprint for r in rows)
def test_an_existing_higher_id_holder_gives_way(backfill, db):
from plugins.endpoints.models import EndpointApplication
assetid = _asset(db, 'FP03')
fingerprint = backfill.compute_fingerprint('Chrome', '1.0', 'Vendor')
older = _row(db, assetid, 'Chrome')
_row(db, assetid, 'Chrome', fingerprint=fingerprint)
summary = backfill.backfill_fingerprints()
assert summary['duplicatesremoved'] == 1
rows = EndpointApplication.query.all()
assert [(r.endpointapplicationid, r.appfingerprint) for r in rows] == \
[(older, fingerprint)]
def test_a_failing_batch_is_rolled_back(backfill, db, monkeypatch):
from plugins.endpoints.models import EndpointApplication
assetid = _asset(db, 'FP04')
_row(db, assetid, 'Chrome')
_row(db, assetid, 'Office')
def broken(apps):
for app in apps:
app.appfingerprint = 'x' * 64
raise RuntimeError('simulated failure')
monkeypatch.setattr(backfill, '_fingerprint_batch', broken)
summary = backfill.backfill_fingerprints()
assert summary['failed'] == 2
assert summary['processed'] == 0
db.session.expire_all()
assert EndpointApplication.query.filter(
EndpointApplication.appfingerprint.isnot(None)).count() == 0

View File

@@ -0,0 +1,109 @@
"""The generated "required on all PCs" policy follows the app by appid.
The generated policy name embeds appname, so a name lookup orphaned the policy
on rename. These pin the appid lookup, rename+untick in one PUT, delete handling
of a customised ('kept') policy and of other policies naming the appid, and the
truthful note that the generated policy is not attached to a business policy.
"""
from shopdb.core.models import Application
def _policies():
from plugins.computers.models import ConfigurationPolicy
return ConfigurationPolicy.query
def _create_required(client, headers, name='Fleet App'):
response = client.post('/api/applications',
json={'appname': name, 'isrequired': True,
'acknowledgepolicy': True},
headers=headers)
assert response.status_code == 201, response.get_json()
return response.get_json()['data']
def test_created_response_says_the_policy_is_not_attached(client, db, auth_headers):
data = _create_required(client, auth_headers)
assert data['policyresult'] == 'created'
assert 'business policy' in data['policynote']
def test_ack_message_mentions_business_policy(client, db, auth_headers):
response = client.post('/api/applications',
json={'appname': 'Unacked', 'isrequired': True},
headers=auth_headers)
assert response.status_code == 400
assert 'business policy' in response.get_json()['data']['error']['message']
def test_rename_keeps_the_policy_and_renames_it(client, db, auth_headers):
data = _create_required(client, auth_headers, 'Old Name')
response = client.put(f"/api/applications/{data['appid']}",
json={'appname': 'New Name'}, headers=auth_headers)
assert response.status_code == 200, response.get_json()
names = [policy.name for policy in _policies().all()]
assert names == ['Required app: New Name']
def test_untick_after_rename_removes_the_policy(client, db, auth_headers):
data = _create_required(client, auth_headers, 'Before')
from plugins.computers.models import ConfigurationPolicy
# Simulate a rename that bypassed the sync (older data): name is stale.
policy = ConfigurationPolicy.query.one()
application = db.session.get(Application, data['appid'])
application.appname = 'After'
db.session.commit()
assert policy.name == 'Required app: Before'
response = client.put(f"/api/applications/{data['appid']}",
json={'isrequired': False}, headers=auth_headers)
assert response.status_code == 200, response.get_json()
assert response.get_json()['data']['policyresult'] == 'removed'
assert _policies().count() == 0
def test_rename_and_untick_in_one_put_removes_the_policy(client, db, auth_headers):
data = _create_required(client, auth_headers, 'Combo')
response = client.put(f"/api/applications/{data['appid']}",
json={'appname': 'Combo Renamed', 'isrequired': False},
headers=auth_headers)
assert response.status_code == 200, response.get_json()
assert response.get_json()['data']['policyresult'] == 'removed'
assert _policies().count() == 0
def test_delete_deactivates_kept_and_referencing_policies(client, db, auth_headers):
from plugins.computers.models import (ConfigurationPolicy,
ConfigurationPolicyAction,
POLICY_SOURCE_LOCAL)
data = _create_required(client, auth_headers, 'Doomed')
generated = ConfigurationPolicy.query.one()
generated.source = POLICY_SOURCE_LOCAL # admin customised it
handmade = ConfigurationPolicy(name='Hand made push', scopejson=None)
db.session.add(handmade)
db.session.flush()
db.session.add(ConfigurationPolicyAction(
configurationpolicyid=handmade.configurationpolicyid,
action='update_app', params={'appid': data['appid']}))
unrelated = ConfigurationPolicy(name='Unrelated', scopejson=None)
db.session.add(unrelated)
db.session.commit()
response = client.delete(f"/api/applications/{data['appid']}",
json={'confirm': 'OK'}, headers=auth_headers)
assert response.status_code == 200, response.get_json()
body = response.get_json()
assert body['data']['policyresult'] == 'kept'
assert sorted(body['data']['deactivatedpolicies']) == \
['Hand made push', 'Required app: Doomed']
db.session.expire_all()
states = {policy.name: policy.isactive for policy in _policies().all()}
assert states == {'Required app: Doomed': False, 'Hand made push': False,
'Unrelated': True}

View File

@@ -121,7 +121,20 @@ EXEMPT_ENDPOINTS = {'knowledgebase.track_click', 'users.update_user',
# resolves to, so the key cannot ack another machine's alert.
# AUTHORING a fleet alert is the actual authority and stays
# permission-gated in the notifications plugin.
'endpoints.acknowledge_fleet_alert'}
'endpoints.acknowledge_fleet_alert',
# Agent enrollment. A PC with no credential asks for one on
# the shared collector key (same shape as the exempt
# collector blueprint); with the shared key switched off it
# takes no auth and only collects a credential an admin
# already approved for that hostname, serial and nonce.
# Approving, rejecting and revoking ARE admin-gated and
# ARE swept.
'endpoints.enroll_endpoint',
# Agent telemetry ingest under the endpoints prefix. The
# same handler as POST /api/collector/endpoints, whose
# blueprint is exempt: authenticated by the per-PC
# credential or the shared collector key.
'endpoints.agent_collect'}
@pytest.fixture(autouse=True)

View File

@@ -0,0 +1,28 @@
"""GET /api/plugins/settings-routes survives a plugin whose hook raises."""
def test_a_raising_plugin_is_skipped_not_fatal(app, client, db, monkeypatch):
manager = app.extensions.get('plugin_manager')
plugins = manager.get_all_plugins()
enabled = [name for name in plugins if manager.registry.is_enabled(name)]
assert len(enabled) >= 2
broken, healthy = enabled[0], enabled[1]
def explode():
raise RuntimeError('broken settings hook')
def contribute():
return [{'hub': 'model-catalog', 'path': 'probe', 'name': 'probe-route',
'tab': 'Probe', 'component': 'probe.vue'}]
monkeypatch.setattr(plugins[broken], 'get_settings_routes', explode,
raising=False)
monkeypatch.setattr(plugins[healthy], 'get_settings_routes', contribute,
raising=False)
response = client.get('/api/plugins/settings-routes')
assert response.status_code == 200, response.get_json()
names = [route['name'] for route in response.get_json()['data']['routes']]
assert 'probe-route' in names

View File

@@ -116,7 +116,11 @@ EXPECTED_HEAD_REVISION['warranty'] = 'warranty0002proof'
# on one endpoint, a caller-authority axis separate from the policy hold.
# 0019 splits the 0007 bundle into one shipped Configuration Policy per monitor
# and drops the shipped Business Policy, so sites compose their own.
EXPECTED_HEAD_REVISION['endpoints'] = 'endpoints0019splitpolicies'
# 0020 adds endpointapplications.appfingerprint for delta app updates. 0021 and
# 0022 are retired no-ops (client certificates, removed). 0023 drops their
# leftover asset columns, renames app_fingerprint, and adds per-PC credentials:
# endpointcredentials, endpointenrollrequests and endpointmigrationwaves.
EXPECTED_HEAD_REVISION['endpoints'] = 'endpoints0023credentials'
# Plugins built after the cutover: their 0001 baseline really creates tables the
# core chain never owned.

View File

@@ -30,6 +30,12 @@ EXEMPT = {
# Written by the app itself when the agent key is minted, not typed.
'plugin.endpoints.agentkeyprovisioned',
'plugin.endpoints.envkeyfallback',
# Edited on the endpoints Enrollment page through
# PUT /api/endpoints/enrollment/settings, not the generic settings view:
# turning the shared key off needs a confirm that shows how many PCs are
# not enrolled yet, which a plain settings field cannot give.
'plugin.endpoints.sharedkeyallowed',
'plugin.endpoints.autoenroll',
}

View File

@@ -192,6 +192,54 @@ def test_an_endpoint_lock_still_blocks_the_upgrade(client, db, app):
assert EndpointCommand.query.filter_by(assetid=assetid).count() == 0
def test_refused_callers_never_trigger_repackaging(client, db, app,
monkeypatch):
"""Lock and runscript checks run BEFORE the package and script are rebuilt.
Repackaging rewrites a shared file and script row; a caller who will be
refused anyway must not be able to cause that side effect.
"""
calls = []
monkeypatch.setattr(agentregistry, 'register_current_agent_package',
lambda: calls.append('package'))
monkeypatch.setattr(agentupgradescript, 'seed_upgrade_script',
lambda: calls.append('script'))
_publish_agent()
assetid = _endpoint(db, 'UPGRADE09')
noscript = _login(db, client, 'noscriptoperator',
permissions=['endpoints.view', 'endpoints.command',
'endpoints.agentupdate'])
response = client.post(UPGRADE_URL.format(assetid), json={},
headers=noscript)
assert response.status_code == 403
assert 'endpoints.runscript' in \
response.get_json()['data']['error']['message']
owner = _login(db, client, 'lockowner2', admin=True)
client.post(f'/api/endpoints/{assetid}/lock',
json={'reason': 'bench repair'}, headers=owner)
locked = _login(db, client, 'lockedoperator',
permissions=['endpoints.view', 'endpoints.command',
'endpoints.agentupdate',
'endpoints.runscript'])
response = client.post(UPGRADE_URL.format(assetid), json={},
headers=locked)
assert response.status_code == 403
assert calls == []
def test_package_write_leaves_no_temp_files(client, db, app, isolate_store):
_publish_agent()
agentregistry.register_current_agent_package()
agentregistry.register_current_agent_package()
leftovers = [name for _root, _dirs, files in
os.walk(isolate_store / 'applicationpackages')
for name in files if name.endswith('.tmp')]
assert leftovers == []
def test_a_second_upgrade_cannot_stack_on_an_in_flight_command(client, db, app):
"""at-most-one-in-flight must still hold through this route."""
from plugins.endpoints.models import EndpointCommand

View File

@@ -5,11 +5,10 @@ including safety features: fingerprint validation, delete count verification,
memory limits, and concurrent update protection.
"""
import pytest
import hashlib
from datetime import datetime
from shopdb.extensions import db
import pytest
from plugins.endpoints.models import EndpointApplication
from plugins.endpoints.plugin import EndpointsPlugin
@@ -23,6 +22,22 @@ def compute_test_fingerprint(name, version, publisher):
return hashlib.sha256(data.encode('utf-8')).hexdigest()
def _make_asset(db, assetnumber):
"""A bare computer-type Asset; _replace_applications needs only the id."""
from shopdb.core.models import Asset, AssetType
assettype = AssetType.query.filter_by(assettype='computer').first()
if assettype is None:
assettype = AssetType(assettype='computer')
db.session.add(assettype)
db.session.flush()
asset = Asset(assetnumber=assetnumber, name=assetnumber,
assettypeid=assettype.assettypeid, isactive=True)
db.session.add(asset)
db.session.commit()
return asset.assetid
class TestApplicationDelta:
"""Test delta-based application inventory updates."""
@@ -31,13 +46,9 @@ class TestApplicationDelta:
return EndpointsPlugin()
@pytest.fixture
def asset_id(self, db_session):
def asset_id(self, db):
"""Create a test asset."""
from shopdb.core.models import Asset
asset = Asset(assettag='TEST-001')
db_session.add(asset)
db_session.commit()
return asset.assetid
return _make_asset(db, 'TEST-001')
def test_fingerprint_validation_valid(self, plugin):
"""Valid fingerprints (64 lowercase hex chars) pass validation."""
@@ -65,7 +76,7 @@ class TestApplicationDelta:
for fp in invalid_fps:
assert not plugin._is_valid_fingerprint(fp), f"Should reject: {fp}"
def test_delta_no_changes(self, plugin, asset_id, db_session):
def test_delta_no_changes(self, plugin, asset_id, db):
"""No apps added/removed when inventory unchanged."""
# Initial insert: 3 apps
apps_v1 = [
@@ -91,7 +102,7 @@ class TestApplicationDelta:
warnings = []
result = plugin._replace_applications(asset_id, apps_v1, warnings)
db_session.commit()
db.session.commit()
assert result == 3
assert len(warnings) == 0
@@ -100,13 +111,13 @@ class TestApplicationDelta:
# Second report: same apps (no changes)
warnings = []
result = plugin._replace_applications(asset_id, apps_v1, warnings)
db_session.commit()
db.session.commit()
assert result == 0 # 0 added
assert len(warnings) == 0
assert EndpointApplication.query.filter_by(assetid=asset_id).count() == 3 # Still 3
def test_delta_add_remove(self, plugin, asset_id, db_session):
def test_delta_add_remove(self, plugin, asset_id, db):
"""Detect adds and removes correctly."""
# Initial: 2 apps
apps_v1 = [
@@ -126,7 +137,7 @@ class TestApplicationDelta:
warnings = []
plugin._replace_applications(asset_id, apps_v1, warnings)
db_session.commit()
db.session.commit()
assert EndpointApplication.query.filter_by(assetid=asset_id).count() == 2
@@ -148,7 +159,7 @@ class TestApplicationDelta:
warnings = []
result = plugin._replace_applications(asset_id, apps_v2, warnings)
db_session.commit()
db.session.commit()
assert result == 1 # 1 added (Office)
assert len(warnings) == 0
@@ -158,7 +169,7 @@ class TestApplicationDelta:
app_names = {app.displayname for app in apps}
assert app_names == {'Chrome', 'Office'}
def test_delta_version_change(self, plugin, asset_id, db_session):
def test_delta_version_change(self, plugin, asset_id, db):
"""Version change detected (different fingerprint)."""
# Initial: Chrome 117.0
apps_v1 = [
@@ -172,7 +183,7 @@ class TestApplicationDelta:
warnings = []
plugin._replace_applications(asset_id, apps_v1, warnings)
db_session.commit()
db.session.commit()
# Update: Chrome 118.0 (different fingerprint)
apps_v2 = [
@@ -186,14 +197,14 @@ class TestApplicationDelta:
warnings = []
result = plugin._replace_applications(asset_id, apps_v2, warnings)
db_session.commit()
db.session.commit()
assert result == 1 # 1 added (new version)
apps = EndpointApplication.query.filter_by(assetid=asset_id).all()
assert len(apps) == 1
assert apps[0].displayversion == '118.0'
def test_invalid_fingerprint_skipped(self, plugin, asset_id, db_session):
def test_invalid_fingerprint_skipped(self, plugin, asset_id, db):
"""Apps with invalid fingerprints are skipped with warning."""
apps = [
{
@@ -212,7 +223,7 @@ class TestApplicationDelta:
warnings = []
result = plugin._replace_applications(asset_id, apps, warnings)
db_session.commit()
db.session.commit()
assert result == 1 # Only Firefox added
assert len(warnings) == 1
@@ -223,7 +234,7 @@ class TestApplicationDelta:
assert len(apps) == 1
assert apps[0].displayname == 'Firefox'
def test_legacy_fallback_no_fingerprints(self, plugin, asset_id, db_session):
def test_legacy_fallback_no_fingerprints(self, plugin, asset_id, db):
"""Legacy mode works when agent doesn't send fingerprints."""
apps = [
{
@@ -242,16 +253,16 @@ class TestApplicationDelta:
warnings = []
result = plugin._replace_applications(asset_id, apps, warnings)
db_session.commit()
db.session.commit()
assert result == 2
assert len(warnings) == 0
apps = EndpointApplication.query.filter_by(assetid=asset_id).all()
assert len(apps) == 2
assert all(app.app_fingerprint is None for app in apps)
assert all(app.appfingerprint is None for app in apps)
def test_empty_inventory(self, plugin, asset_id, db_session):
def test_empty_inventory(self, plugin, asset_id, db):
"""Empty inventory removes all apps."""
# Initial: 2 apps
apps_v1 = [
@@ -271,19 +282,19 @@ class TestApplicationDelta:
warnings = []
plugin._replace_applications(asset_id, apps_v1, warnings)
db_session.commit()
db.session.commit()
assert EndpointApplication.query.filter_by(assetid=asset_id).count() == 2
# Update: Empty list
warnings = []
result = plugin._replace_applications(asset_id, [], warnings)
db_session.commit()
db.session.commit()
assert result == 0
assert EndpointApplication.query.filter_by(assetid=asset_id).count() == 0
def test_missing_displayname_skipped(self, plugin, asset_id, db_session):
def test_missing_displayname_skipped(self, plugin, asset_id, db):
"""Apps without displayname are skipped."""
apps = [
{
@@ -302,13 +313,13 @@ class TestApplicationDelta:
warnings = []
result = plugin._replace_applications(asset_id, apps, warnings)
db_session.commit()
db.session.commit()
assert result == 1 # Only Firefox added
assert len(warnings) == 1
assert 'no displayname' in warnings[0].lower()
def test_string_truncation(self, plugin, asset_id, db_session):
def test_string_truncation(self, plugin, asset_id, db):
"""Long strings are truncated to column limits."""
apps = [
{
@@ -321,7 +332,7 @@ class TestApplicationDelta:
warnings = []
result = plugin._replace_applications(asset_id, apps, warnings)
db_session.commit()
db.session.commit()
assert result == 1
app = EndpointApplication.query.filter_by(assetid=asset_id).first()
@@ -338,13 +349,9 @@ class TestApplicationDeltaSafety:
return EndpointsPlugin()
@pytest.fixture
def asset_id(self, db_session):
def asset_id(self, db):
"""Create a test asset."""
from shopdb.core.models import Asset
asset = Asset(assettag='TEST-SAFETY-001')
db_session.add(asset)
db_session.commit()
return asset.assetid
return _make_asset(db, 'TEST-SAFETY-001')
def test_compute_fingerprint_consistency(self):
"""Fingerprint algorithm is deterministic."""

View File

@@ -0,0 +1,107 @@
"""collect_log sensitive-path denylist at enqueue.
The agent refuses answer files, its own install/state and key material. The
server refuses the same set at enqueue so the operator gets a 400 naming the
reason instead of a queued command that fails on the endpoint.
"""
import pytest
from werkzeug.security import generate_password_hash
from plugins.endpoints.api.routes import _validate_log_path
@pytest.mark.parametrize('path', [
r'C:\Windows\Panther\setupact.log',
r'c:/windows/panther/UnattendGC/setupact.log',
r'%SystemRoot%\Panther\setupact.log',
r'%windir%\System32\Sysprep\Panther\setupact.log',
r'D:\WINDOWS\System32\sysprep\x.log',
r'C:\Windows\\Panther\.\setupact.log',
r'C:\ProgramData\ShopDB\state.json',
r'%ProgramData%\shopdb\agent.log',
r'C:\Program Files\ShopDB\EndpointAgent\agent.log',
r'C:\PROGRA~1\ShopDB\EndpointAgent\agent.log',
r'C:\temp\Unattend.xml',
r'C:\temp\unattend-original.xml',
r'C:\temp\AutoUnattend.xml',
r'C:\app\appsettings.json',
r'C:\app\AppSettings.Production.json',
r'C:\certs\server.KEY',
r'C:\certs\client.pfx',
r'C:\certs\client.p12',
])
def test_sensitive_paths_are_refused_with_a_reason(path):
error = _validate_log_path(path)
assert error is not None
assert 'not permitted' in error
@pytest.mark.parametrize('path', [
r'C:\Windows\Logs\CBS\CBS.log',
r'C:\ProgramData\Vendor\app.log',
r'C:\Program Files\Other\settings.json',
r'C:\temp\notunattend.xml',
])
def test_ordinary_log_paths_still_pass(path):
assert _validate_log_path(path) is None
def _endpoint(db, hostname):
from shopdb.api import Asset, AssetType
from plugins.computers.models import Computer
from plugins.endpoints.models import EndpointState
assettype = AssetType.query.filter_by(assettype='computer').first()
if assettype is None:
assettype = AssetType(assettype='computer', pluginname='computers',
tablename='computers')
db.session.add(assettype)
db.session.flush()
asset = Asset(assetnumber=hostname, name=hostname,
assettypeid=assettype.assettypeid, isactive=True)
db.session.add(asset)
db.session.flush()
db.session.add(Computer(assetid=asset.assetid, hostname=hostname))
db.session.add(EndpointState(assetid=asset.assetid, hostname=hostname,
agentversion='1.16.0.0'))
db.session.commit()
return asset.assetid
def _admin(db, client, username='denylistadmin'):
from shopdb.core.models import User, Role, Permission
Permission.seed()
db.session.flush()
role = Role.query.filter_by(rolename='admin').first()
if role is None:
role = Role(rolename='admin')
db.session.add(role)
db.session.commit()
user = User(username=username, email=username + '@test.local',
passwordhash=generate_password_hash('testpass'))
user.roles.append(role)
db.session.add(user)
db.session.commit()
response = client.post('/api/auth/login',
json={'username': username, 'password': 'testpass'})
token = response.get_json()['data']['access_token']
return {'Authorization': f'Bearer {token}'}
def test_enqueue_refuses_an_answer_file_and_queues_nothing(app, client, db):
from plugins.endpoints.models import EndpointCommand
assetid = _endpoint(db, 'denylistbox')
headers = _admin(db, client)
response = client.post(f'/api/endpoints/{assetid}/commands',
json={'action': 'collect_log',
'target': r'%SystemRoot%\Panther\unattend.xml'},
headers=headers)
assert response.status_code == 400
message = response.get_json()['data']['error']['message']
assert 'Panther' in message
assert EndpointCommand.query.filter_by(assetid=assetid).count() == 0

View File

@@ -0,0 +1,589 @@
"""Per-PC endpoint credentials: enrollment, authentication and identity binding.
A PC enrolls once on the shared fleet key and gets its own secret. From then on
the server derives the calling asset from that secret, so a request naming a
different PC is refused (ASSET_MISMATCH), the shared key is refused for that PC
(SHARED_KEY_REFUSED), and a revoked secret is refused outright
(CREDENTIAL_INVALID) so the agent knows to enroll again.
"""
import pytest
from shopdb.core.models import Asset, AssetType, Setting
COLLECTOR_KEY = 'testcollectorkey'
ENROLL_URL = '/api/endpoints/enroll'
NONCE = 'n' * 40
OTHER_NONCE = 'm' * 40
@pytest.fixture
def keyheaders(app):
app.config['COLLECTOR_API_KEY'] = COLLECTOR_KEY
return {'X-API-Key': COLLECTOR_KEY}
def _computer(db, hostname, serialnumber='SER-1'):
from plugins.computers.models import Computer
atype = AssetType.query.filter_by(assettype='computer').first()
if atype is None:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
serialnumber=serialnumber, isactive=True)
db.session.add(asset)
db.session.flush()
db.session.add(Computer(assetid=asset.assetid, hostname=hostname))
db.session.commit()
return asset.assetid
def _enroll_body(hostname='BAY-01', serialnumber='SER-1', nonce=NONCE):
return {'hostname': hostname, 'serialnumber': serialnumber,
'deviceid': None, 'enrollnonce': nonce, 'agentversion': '1.20.0'}
def _enroll(client, headers, **kwargs):
return client.post(ENROLL_URL, json=_enroll_body(**kwargs), headers=headers)
def _credential(client, keyheaders, **kwargs):
response = _enroll(client, keyheaders, **kwargs)
assert response.status_code == 201, response.get_data(as_text=True)
return response.get_json()['data']['credential']
def _setting(db, key, value):
Setting.set(key, value, valuetype='boolean', category='plugin.endpoints')
db.session.commit()
def _code(response):
return response.get_json()['data']['error']['code']
# -- enroll ------------------------------------------------------------------
def test_auto_enroll_issues_a_credential_once(client, db, app, keyheaders):
from plugins.endpoints.models import EndpointCredential
assetid = _computer(db, 'BAY-01')
response = _enroll(client, keyheaders)
assert response.status_code == 201, response.get_data(as_text=True)
data = response.get_json()['data']
assert data['status'] == 'enrolled'
assert data['assetid'] == assetid
assert data['credential'].startswith('sdbe_')
assert len(data['credential']) == 48
row = EndpointCredential.query.filter_by(assetid=assetid).one()
assert row.enrolledvia == 'auto'
assert row.credentialprefix == data['credential'][:12]
# Hash only, never the secret.
assert data['credential'] not in (row.credentialhash, row.credentialprefix)
again = _enroll(client, keyheaders)
assert again.status_code == 202
assert again.get_json()['data']['reason'] == 'already enrolled'
@pytest.mark.parametrize('setup,body,reason', [
('none', {}, 'unknown hostname'),
('duplicate', {}, 'duplicate hostname'),
('noserial', {}, 'no serial on record'),
('serial', {'serialnumber': 'OTHER'}, 'serial mismatch'),
('noauto', {}, 'awaiting approval'),
])
def test_each_pending_reason(client, db, app, keyheaders, setup, body, reason):
if setup == 'duplicate':
_computer(db, 'BAY-01')
_computer(db, 'bay-01')
elif setup == 'noserial':
_computer(db, 'BAY-01', serialnumber=None)
elif setup in ('serial', 'noauto'):
_computer(db, 'BAY-01')
if setup == 'noauto':
_setting(db, 'plugin.endpoints.autoenroll', False)
response = _enroll(client, keyheaders, **body)
assert response.status_code == 202, response.get_data(as_text=True)
data = response.get_json()['data']
assert data['status'] == 'pending'
assert data['reason'] == reason
assert data['requestid']
def test_serial_compare_ignores_case_and_whitespace(client, db, app, keyheaders):
_computer(db, 'BAY-01', serialnumber=' ser-1 ')
assert _enroll(client, keyheaders, serialnumber='SER-1').status_code == 201
def test_enroll_validates_the_nonce(client, db, app, keyheaders):
_computer(db, 'BAY-01')
for nonce in ('short', 'x' * 129, 'bad nonce ' * 4):
response = _enroll(client, keyheaders, nonce=nonce)
assert response.status_code == 400, nonce
def test_enroll_requires_the_shared_key_while_it_is_allowed(client, db, app,
keyheaders):
_computer(db, 'BAY-01')
response = client.post(ENROLL_URL, json=_enroll_body())
assert response.status_code in (401, 500)
def test_a_second_nonce_on_a_pending_request_stays_pending(client, db, app,
keyheaders):
from plugins.endpoints.models import EndpointEnrollRequest
_computer(db, 'BAY-01')
_setting(db, 'plugin.endpoints.autoenroll', False)
first = _enroll(client, keyheaders)
assert first.status_code == 202
# Auto-enroll back on: a different installation still does not get through.
_setting(db, 'plugin.endpoints.autoenroll', True)
second = _enroll(client, keyheaders, nonce=OTHER_NONCE)
assert second.status_code == 202
assert second.get_json()['data']['reason'] == 'nonce changed'
row = db.session.get(EndpointEnrollRequest,
first.get_json()['data']['requestid'])
assert row.enrollnonce == NONCE
def test_approved_request_issues_only_to_its_nonce(client, db, app, keyheaders,
auth_headers):
from plugins.endpoints.models import EndpointCredential
assetid = _computer(db, 'BAY-01')
_setting(db, 'plugin.endpoints.autoenroll', False)
pending = _enroll(client, keyheaders)
requestid = pending.get_json()['data']['requestid']
approve = client.post(
f'/api/endpoints/enrollment/requests/{requestid}/approve',
json={'assetid': assetid}, headers=auth_headers)
assert approve.status_code == 200, approve.get_data(as_text=True)
wrong = _enroll(client, keyheaders, nonce=OTHER_NONCE)
assert wrong.status_code == 403
assert _code(wrong) == 'ENROLL_NONCE_MISMATCH'
right = _enroll(client, keyheaders)
assert right.status_code == 201, right.get_data(as_text=True)
assert EndpointCredential.query.filter_by(assetid=assetid).one() \
.enrolledvia == 'approved'
def test_shared_key_off_enroll_takes_no_auth_but_never_auto_enrolls(
client, db, app, keyheaders, auth_headers):
assetid = _computer(db, 'BAY-01')
_setting(db, 'plugin.endpoints.sharedkeyallowed', False)
pending = client.post(ENROLL_URL, json=_enroll_body())
assert pending.status_code == 202, pending.get_data(as_text=True)
assert pending.get_json()['data']['reason'] == 'awaiting approval'
requestid = pending.get_json()['data']['requestid']
client.post(f'/api/endpoints/enrollment/requests/{requestid}/approve',
json={'assetid': assetid}, headers=auth_headers)
issued = client.post(ENROLL_URL, json=_enroll_body())
assert issued.status_code == 201
def test_rejected_request_stays_rejected_for_the_same_nonce(
client, db, app, keyheaders, auth_headers):
_computer(db, 'BAY-01', serialnumber='SER-9')
pending = _enroll(client, keyheaders)
requestid = pending.get_json()['data']['requestid']
client.post(f'/api/endpoints/enrollment/requests/{requestid}/reject',
headers=auth_headers)
again = _enroll(client, keyheaders)
assert again.status_code == 202
assert again.get_json()['data']['reason'] == 'rejected'
# -- agent auth --------------------------------------------------------------
def test_shared_key_is_refused_after_enrollment(client, db, app, keyheaders):
_computer(db, 'BAY-01')
secret = _credential(client, keyheaders)
refused = client.get('/api/endpoints/alerts?hostname=BAY-01',
headers=keyheaders)
assert refused.status_code == 401
assert _code(refused) == 'SHARED_KEY_REFUSED'
ok = client.get('/api/endpoints/alerts',
headers={'X-Endpoint-Credential': secret})
assert ok.status_code == 200, ok.get_data(as_text=True)
def test_shared_key_still_works_for_a_pc_that_has_not_enrolled(
client, db, app, keyheaders):
_computer(db, 'BAY-01')
_computer(db, 'BAY-02')
_credential(client, keyheaders)
response = client.get('/api/endpoints/alerts?hostname=BAY-02',
headers=keyheaders)
assert response.status_code == 200
def test_shared_key_switched_off_is_refused(client, db, app, keyheaders):
_computer(db, 'BAY-01')
_setting(db, 'plugin.endpoints.sharedkeyallowed', False)
response = client.get('/api/endpoints/alerts?hostname=BAY-01',
headers=keyheaders)
assert response.status_code == 401
assert _code(response) == 'SHARED_KEY_REFUSED'
def test_credential_is_invalid_after_revoke(client, db, app, keyheaders,
auth_headers):
assetid = _computer(db, 'BAY-01')
secret = _credential(client, keyheaders)
revoke = client.post(f'/api/endpoints/enrollment/assets/{assetid}/revoke',
headers=auth_headers)
assert revoke.status_code == 200, revoke.get_data(as_text=True)
response = client.get('/api/endpoints/alerts',
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 401
assert _code(response) == 'CREDENTIAL_INVALID'
# Revoked: the shared key is admitted again and the PC can re-enroll.
assert client.get('/api/endpoints/alerts?hostname=BAY-01',
headers=keyheaders).status_code == 200
reissued = _credential(client, keyheaders)
assert reissued != secret
def test_garbage_credential_is_invalid(client, db, app):
response = client.get('/api/endpoints/commands/next',
headers={'X-Endpoint-Credential': 'sdbe_nope'})
assert response.status_code == 401
assert _code(response) == 'CREDENTIAL_INVALID'
def test_package_downloads_need_no_identity(client, db, app, keyheaders):
_computer(db, 'BAY-01')
secret = _credential(client, keyheaders)
response = client.get('/api/endpoints/scriptbody/999',
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 404
# -- ASSET_MISMATCH ----------------------------------------------------------
def test_commands_next_refuses_another_hostname(client, db, app, keyheaders):
_computer(db, 'BAY-01')
_computer(db, 'BAY-02', serialnumber='SER-2')
secret = _credential(client, keyheaders)
response = client.get('/api/endpoints/commands/next?hostname=BAY-02',
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 403
assert _code(response) == 'ASSET_MISMATCH'
def test_commands_next_claims_for_the_credential_asset(client, db, app,
keyheaders):
from plugins.endpoints.models import EndpointCommand
assetid = _computer(db, 'BAY-01')
secret = _credential(client, keyheaders)
db.session.add(EndpointCommand(assetid=assetid, action='mtconnect_status',
status=EndpointCommand.STATUS_PENDING))
db.session.commit()
response = client.get('/api/endpoints/commands/next',
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 200, response.get_data(as_text=True)
assert response.get_json()['data']['action'] == 'mtconnect_status'
def test_command_result_refuses_another_assets_command(client, db, app,
keyheaders):
from plugins.endpoints.models import EndpointCommand
_computer(db, 'BAY-01')
other = _computer(db, 'BAY-02', serialnumber='SER-2')
secret = _credential(client, keyheaders)
command = EndpointCommand(assetid=other, action='mtconnect_status',
status=EndpointCommand.STATUS_RUNNING)
db.session.add(command)
db.session.commit()
response = client.post(
f'/api/endpoints/commands/{command.endpointcommandid}/result',
json={'succeeded': True},
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 403
assert _code(response) == 'ASSET_MISMATCH'
def test_command_result_for_own_command_needs_no_hostname(client, db, app,
keyheaders):
from plugins.endpoints.models import EndpointCommand
assetid = _computer(db, 'BAY-01')
secret = _credential(client, keyheaders)
command = EndpointCommand(assetid=assetid, action='mtconnect_status',
status=EndpointCommand.STATUS_RUNNING)
db.session.add(command)
db.session.commit()
response = client.post(
f'/api/endpoints/commands/{command.endpointcommandid}/result',
json={'succeeded': True},
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 200, response.get_data(as_text=True)
def _fleet_alert(db):
from plugins.notifications.models import Notification, NotificationType
ntype = NotificationType(typename='Fleet Alert', typecolor='#d93f3f',
isactive=True)
db.session.add(ntype)
db.session.flush()
note = Notification(notificationtypeid=ntype.notificationtypeid,
notification='Line 3 is down', alertfleet=True,
alerttimeout=60, isactive=True)
db.session.add(note)
db.session.commit()
return note.notificationid
def test_alert_ack_refuses_another_asset(client, db, app, keyheaders):
_computer(db, 'BAY-01')
other = _computer(db, 'BAY-02', serialnumber='SER-2')
secret = _credential(client, keyheaders)
notificationid = _fleet_alert(db)
response = client.post(
f'/api/endpoints/alerts/{notificationid}/acknowledge',
json={'assetid': other},
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 403
assert _code(response) == 'ASSET_MISMATCH'
byhostname = client.post(
f'/api/endpoints/alerts/{notificationid}/acknowledge',
json={'hostname': 'BAY-02'},
headers={'X-Endpoint-Credential': secret})
assert byhostname.status_code == 403
assert _code(byhostname) == 'ASSET_MISMATCH'
def test_alert_ack_with_no_assetid_means_the_caller(client, db, app,
keyheaders):
from plugins.notifications.models import NotificationAcknowledgement
assetid = _computer(db, 'BAY-01')
secret = _credential(client, keyheaders)
notificationid = _fleet_alert(db)
response = client.post(
f'/api/endpoints/alerts/{notificationid}/acknowledge',
json={'assetid': 0},
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 201, response.get_data(as_text=True)
assert NotificationAcknowledgement.query.filter_by(
notificationid=notificationid, assetid=assetid).count() == 1
def test_collector_computers_refuses_another_hostname(client, db, app,
keyheaders):
_computer(db, 'BAY-01')
_computer(db, 'BAY-02', serialnumber='SER-2')
secret = _credential(client, keyheaders)
headers = {'X-Endpoint-Credential': secret}
mismatch = client.post('/api/collector/computers',
json={'hostname': 'BAY-02'}, headers=headers)
assert mismatch.status_code == 403, mismatch.get_data(as_text=True)
assert _code(mismatch) == 'ASSET_MISMATCH'
# A credential never creates a PC.
unknown = client.post('/api/collector/computers',
json={'hostname': 'NEW-BAY'}, headers=headers)
assert unknown.status_code == 403
assert _code(unknown) == 'ASSET_MISMATCH'
own = client.post('/api/collector/computers',
json={'hostname': 'BAY-01'}, headers=headers)
assert own.status_code == 200, own.get_data(as_text=True)
def test_collector_refuses_shared_key_for_an_enrolled_pc(client, db, app,
keyheaders):
_computer(db, 'BAY-01')
_credential(client, keyheaders)
response = client.post('/api/collector/computers',
json={'hostname': 'BAY-01'}, headers=keyheaders)
assert response.status_code == 401
assert _code(response) == 'SHARED_KEY_REFUSED'
def test_collector_invalid_credential(client, db, app):
response = client.post('/api/collector/printers',
json={'hostname': 'BAY-01', 'queues': []},
headers={'X-Endpoint-Credential': 'sdbe_' + 'a' * 43})
assert response.status_code == 401
assert _code(response) == 'CREDENTIAL_INVALID'
def test_endpoints_collect_route_is_the_collector(client, db, app, keyheaders):
from plugins.endpoints.models import EndpointState
assetid = _computer(db, 'BAY-01')
secret = _credential(client, keyheaders)
response = client.post('/api/endpoints/collect',
json={'hostname': 'BAY-01', 'agentversion': '1.20.0'},
headers={'X-Endpoint-Credential': secret})
assert response.status_code == 200, response.get_data(as_text=True)
assert EndpointState.query.filter_by(assetid=assetid).one() \
.agentversion == '1.20.0'
_computer(db, 'BAY-02', serialnumber='SER-2')
mismatch = client.post('/api/endpoints/collect',
json={'hostname': 'BAY-02'},
headers={'X-Endpoint-Credential': secret})
assert mismatch.status_code == 403
assert _code(mismatch) == 'ASSET_MISMATCH'
def test_contract_exports_answer_through_the_endpoints_plugin(client, db, app,
keyheaders):
from shopdb.api import agent_shared_key_refused, resolve_agent_credential
assetid = _computer(db, 'BAY-01')
assert agent_shared_key_refused('BAY-01') is False
secret = _credential(client, keyheaders)
with app.test_request_context():
assert resolve_agent_credential(secret) == assetid
assert resolve_agent_credential('sdbe_' + 'z' * 43) is None
assert agent_shared_key_refused('BAY-01') is True
assert agent_shared_key_refused('UNKNOWN') is False
# -- admin gating ------------------------------------------------------------
@pytest.mark.parametrize('method,url', [
('get', '/api/endpoints/enrollment'),
('get', '/api/endpoints/enrollment/migration'),
('post', '/api/endpoints/enrollment/requests/1/approve'),
('post', '/api/endpoints/enrollment/requests/1/reject'),
('post', '/api/endpoints/enrollment/assets/1/revoke'),
('put', '/api/endpoints/enrollment/settings'),
('post', '/api/endpoints/enrollment/migration/wave'),
])
def test_member_without_a_role_is_forbidden(client, db, member_headers, method,
url):
response = getattr(client, method)(url, json={}, headers=member_headers)
assert response.status_code == 403
def test_enrollment_listing_and_settings(client, db, app, keyheaders,
auth_headers):
enrolled = _computer(db, 'BAY-01')
_computer(db, 'BAY-02', serialnumber='SER-2')
_computer(db, 'BAY-03', serialnumber=None)
_credential(client, keyheaders)
_enroll(client, keyheaders, hostname='BAY-03', serialnumber='X')
response = client.get('/api/endpoints/enrollment', headers=auth_headers)
assert response.status_code == 200, response.get_data(as_text=True)
body = response.get_json()
data = body['data']
assert data['settings'] == {'sharedkeyallowed': True, 'autoenroll': True}
assert data['counts'] == {'enrolled': 1, 'notenrolled': 1, 'pending': 1}
states = {row['hostname']: row['state'] for row in data['computers']}
assert states == {'BAY-01': 'enrolled', 'BAY-02': 'notenrolled',
'BAY-03': 'pending'}
row = next(r for r in data['computers'] if r['assetid'] == enrolled)
assert row['credentialprefix'].startswith('sdbe_')
assert row['enrolledvia'] == 'auto'
assert [r['reason'] for r in data['requests']
if r['status'] == 'pending'] == ['no serial on record']
assert body['meta']['pagination']['total'] == 3
filtered = client.get('/api/endpoints/enrollment?state=enrolled',
headers=auth_headers).get_json()['data']
assert [r['hostname'] for r in filtered['computers']] == ['BAY-01']
put = client.put('/api/endpoints/enrollment/settings',
json={'sharedkeyallowed': False}, headers=auth_headers)
assert put.status_code == 200
assert put.get_json()['data'] == {'sharedkeyallowed': False,
'autoenroll': True}
bad = client.put('/api/endpoints/enrollment/settings',
json={'autoenroll': 'yes'}, headers=auth_headers)
assert bad.status_code == 400
def test_approve_refuses_an_enrolled_computer(client, db, app, keyheaders,
auth_headers):
assetid = _computer(db, 'BAY-01')
_credential(client, keyheaders)
pending = _enroll(client, keyheaders, nonce=OTHER_NONCE)
requestid = pending.get_json()['data']['requestid']
response = client.post(
f'/api/endpoints/enrollment/requests/{requestid}/approve',
json={'assetid': assetid}, headers=auth_headers)
assert response.status_code == 409
# -- server certificate pin --------------------------------------------------
def test_server_cert_reports_the_spki_pin(client, db, auth_headers, tmp_path,
monkeypatch):
import hashlib
from datetime import datetime, timedelta, timezone
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.x509.oid import NameOID
from plugins.endpoints.api import routes as endpoint_routes
key = ec.generate_private_key(ec.SECP256R1())
name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, 'pin.example.test')])
now = datetime.now(timezone.utc)
certificate = (x509.CertificateBuilder()
.subject_name(name).issuer_name(name)
.public_key(key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now - timedelta(days=1))
.not_valid_after(now + timedelta(days=30))
.sign(key, hashes.SHA256()))
certdir = tmp_path / 'certs'
certdir.mkdir()
(certdir / 'server.crt').write_bytes(
certificate.public_bytes(serialization.Encoding.PEM))
monkeypatch.chdir(tmp_path)
if hasattr(endpoint_routes, '_server_cert_dir'):
monkeypatch.setattr(endpoint_routes, '_server_cert_dir', lambda: certdir)
expected = hashlib.sha256(key.public_key().public_bytes(
serialization.Encoding.DER,
serialization.PublicFormat.SubjectPublicKeyInfo)).hexdigest()
response = client.get('/api/endpoints/settings/server-cert',
headers=auth_headers)
assert response.status_code == 200, response.get_data(as_text=True)
assert response.get_json()['data']['spkisha256'] == expected

View File

@@ -0,0 +1,139 @@
"""Migration 0023: per-PC credential tables, and the client certificate cleanup.
Two starting points matter. A database that ran the 2026-09-14 builds of
0020-0022 carries app_fingerprint and five certificate columns on assets; a
fresh one carries neither. Both must reach the same schema, twice over.
"""
import importlib.util
from pathlib import Path
from alembic.migration import MigrationContext
from alembic.operations import Operations
from sqlalchemy import create_engine, inspect, text
_VERSIONS = (Path(__file__).resolve().parent.parent.parent
/ 'plugins' / 'endpoints' / 'migrations' / 'versions')
_CHAIN = ('0020_app_delta_fingerprints.py', '0021_client_certificates.py',
'0022_client_cert_revocation.py', '0023_endpoint_credentials.py')
_CERT_COLUMNS = {'client_cert_fingerprint', 'cert_issued_at', 'cert_expires_at',
'cert_revoked_at', 'cert_revocation_reason'}
def _load(filename):
path = _VERSIONS / filename
spec = importlib.util.spec_from_file_location('endpoints_mig_' + path.stem,
str(path))
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _run(connection, step='upgrade', files=_CHAIN):
operations = Operations(MigrationContext.configure(connection))
operations._install_proxy()
try:
for filename in files:
getattr(_load(filename), step)()
finally:
operations._remove_proxy()
connection.commit()
def _base_schema(connection, legacy):
certcolumns = ''
if legacy:
certcolumns = (', client_cert_fingerprint VARCHAR(64), '
'cert_issued_at DATETIME, cert_expires_at DATETIME, '
'cert_revoked_at DATETIME, '
'cert_revocation_reason VARCHAR(255)')
connection.execute(text(
'CREATE TABLE assets (assetid INTEGER PRIMARY KEY, '
'assetnumber VARCHAR(50)' + certcolumns + ')'))
fingerprint = ', app_fingerprint VARCHAR(64)' if legacy else ''
connection.execute(text(
'CREATE TABLE endpointapplications (endpointapplicationid INTEGER '
'PRIMARY KEY, assetid INTEGER, displayname VARCHAR(255)'
+ fingerprint + ')'))
if legacy:
connection.execute(text(
'CREATE INDEX idx_assets_cert_fingerprint ON assets '
'(client_cert_fingerprint)'))
connection.execute(text(
'CREATE UNIQUE INDEX idx_endpointapplications_fingerprint ON '
'endpointapplications (assetid, app_fingerprint)'))
connection.execute(text(
"INSERT INTO endpointapplications VALUES (1, 7, 'App', '"
+ 'a' * 64 + "')"))
connection.commit()
def _assert_head_schema(connection):
inspector = inspect(connection)
assetcolumns = {c['name'] for c in inspector.get_columns('assets')}
assert not assetcolumns & _CERT_COLUMNS
appcolumns = {c['name'] for c in inspector.get_columns('endpointapplications')}
assert 'appfingerprint' in appcolumns
assert 'app_fingerprint' not in appcolumns
indexed = {tuple(ix['column_names']): ix['unique'] for ix in
inspector.get_indexes('endpointapplications')}
assert indexed.get(('assetid', 'appfingerprint')) in (True, 1)
for table in ('endpointcredentials', 'endpointenrollrequests',
'endpointmigrationwaves'):
assert inspector.has_table(table), table
credentialcolumns = {c['name'] for c in inspector.get_columns('endpointcredentials')}
assert credentialcolumns == {
'credentialid', 'assetid', 'credentialhash', 'credentialprefix',
'enrolledvia', 'issuedat', 'lastusedat', 'revokedat', 'revokedby'}
requestindexes = [tuple(ix['column_names']) for ix in
inspector.get_indexes('endpointenrollrequests')]
assert requestindexes.count(('hostname', 'serialnumber')) == 1
def test_upgrade_from_the_client_certificate_builds():
engine = create_engine('sqlite://')
with engine.connect() as connection:
_base_schema(connection, legacy=True)
_run(connection)
_assert_head_schema(connection)
# Data survives the rename.
assert connection.execute(text(
'SELECT appfingerprint FROM endpointapplications')).scalar() == 'a' * 64
_run(connection)
_assert_head_schema(connection)
def test_upgrade_on_a_fresh_database():
engine = create_engine('sqlite://')
with engine.connect() as connection:
_base_schema(connection, legacy=False)
_run(connection)
_assert_head_schema(connection)
_run(connection)
_assert_head_schema(connection)
def test_retired_certificate_revisions_are_no_ops():
engine = create_engine('sqlite://')
with engine.connect() as connection:
_base_schema(connection, legacy=False)
_run(connection, files=_CHAIN[1:3])
columns = {c['name'] for c in inspect(connection).get_columns('assets')}
assert not columns & _CERT_COLUMNS
def test_downgrade_drops_the_credential_tables():
engine = create_engine('sqlite://')
with engine.connect() as connection:
_base_schema(connection, legacy=False)
_run(connection)
_run(connection, step='downgrade', files=_CHAIN[3:])
inspector = inspect(connection)
for table in ('endpointcredentials', 'endpointenrollrequests',
'endpointmigrationwaves'):
assert not inspector.has_table(table), table
_run(connection, files=_CHAIN[3:])
_assert_head_schema(connection)

View File

@@ -0,0 +1,278 @@
"""Moving the existing fleet onto per-PC credentials.
A PC on agent 1.19 or older holds only the shared key. It moves by receiving
agent 1.20, which enrolls itself. The migration surface computes, per computer,
whether that upgrade can be pushed, and pushes it in waves through the same
enqueue path as the single-asset agentupgrade route, packaging once per wave.
The agent store is isolated per test, as in test_endpoints_agent_upgrade_route.
"""
from datetime import datetime, timedelta, timezone
import pytest
from plugins.endpoints.api import agentregistry, agentstore, agentupgradescript
MIGRATION_URL = '/api/endpoints/enrollment/migration'
WAVE_URL = '/api/endpoints/enrollment/migration/wave'
_PE_BYTES = b'MZ' + b'\x00' * 4096
@pytest.fixture(autouse=True)
def isolate_store(tmp_path, monkeypatch):
siteagent = tmp_path / 'siteagent'
packages = tmp_path / 'applicationpackages'
scripts = tmp_path / 'scripts'
for directory in (siteagent, packages, scripts):
directory.mkdir()
monkeypatch.setattr(agentstore, '_siteagent_dir', lambda: str(siteagent))
monkeypatch.setattr(agentregistry, 'application_package_location',
lambda: (str(packages), '/api/applications/package/'))
monkeypatch.setattr(agentupgradescript, 'script_body_location',
lambda: (str(scripts), '/api/scripts/body/'))
return tmp_path
def _now():
return datetime.now(timezone.utc).replace(tzinfo=None)
def _publish_agent(version='1.20.0'):
agentstore.store_version('agent', version, _PE_BYTES, make_current=True)
def _pc(db, hostname, agentversion='1.19.0', serialnumber='SER',
reportedago=timedelta(hours=1), reports=True):
from shopdb.api import Asset, AssetType
from plugins.computers.models import Computer
from plugins.endpoints.models import EndpointState
atype = AssetType.query.filter_by(assettype='computer').first()
if atype is None:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
asset = Asset(assetnumber=hostname + '-asset', name=hostname,
assettypeid=atype.assettypeid, isactive=True,
serialnumber=serialnumber)
db.session.add(asset)
db.session.flush()
db.session.add(Computer(assetid=asset.assetid, hostname=hostname))
if reports:
db.session.add(EndpointState(
assetid=asset.assetid, hostname=hostname, agentversion=agentversion,
lastreportedat=_now() - reportedago if reportedago else None))
db.session.commit()
return asset.assetid
def _rows(client, headers, query=''):
response = client.get(MIGRATION_URL + query, headers=headers)
assert response.status_code == 200, response.get_data(as_text=True)
return response.get_json()['data']
def _by_host(data):
return {row['hostname']: row for row in data['computers']}
def test_readiness_states(client, db, app, auth_headers):
from plugins.endpoints.api.agentauth import generate_credential
from plugins.endpoints.api.enrollment import issue_credential
from plugins.endpoints.models import EndpointCommand, EndpointEnrollRequest
_publish_agent()
enrolled = _pc(db, 'PC-ENROLLED')
issue_credential(enrolled, 'auto')
pending = _pc(db, 'PC-PENDING')
db.session.add(EndpointEnrollRequest(
hostname='pc-pending', serialnumber='SER', enrollnonce=generate_credential(),
status='pending', reason='awaiting approval', matchedassetid=pending,
requestedat=_now(), lastseenat=_now()))
upgrading = _pc(db, 'PC-UPGRADING')
_pc(db, 'PC-READY')
_pc(db, 'PC-BLOCKED', serialnumber=None, reportedago=timedelta(days=3))
_pc(db, 'PC-SHAREONLY', reports=False)
_pc(db, 'PC-NEW', agentversion='1.20.0.0')
db.session.commit()
seeded_app = agentregistry.register_current_agent_package()
assert seeded_app is not None
scriptid = agentupgradescript.seed_upgrade_script()['scriptid']
db.session.add(EndpointCommand(assetid=upgrading, action='run_script',
target=f'script:{scriptid}',
status=EndpointCommand.STATUS_PENDING))
db.session.commit()
data = _rows(client, auth_headers)
rows = _by_host(data)
assert data['target'] == '1.20.0'
assert rows['PC-ENROLLED']['state'] == 'enrolled'
assert rows['PC-PENDING']['state'] == 'pending'
assert rows['PC-UPGRADING']['state'] == 'upgrading'
assert rows['PC-READY']['state'] == 'ready'
assert rows['PC-READY']['blockers'] == []
assert rows['PC-BLOCKED']['state'] == 'blocked'
assert rows['PC-BLOCKED']['blockers'] == ['offline', 'no serial on record']
assert rows['PC-SHAREONLY']['state'] == 'notreporting'
assert rows['PC-NEW']['state'] == 'blocked'
assert rows['PC-NEW']['blockers'] == ['awaiting enrollment']
assert data['counts'] == {'enrolled': 1, 'pending': 1, 'upgrading': 1,
'ready': 1, 'blocked': 2, 'notreporting': 1}
only = _rows(client, auth_headers, '?state=ready')
assert [row['hostname'] for row in only['computers']] == ['PC-READY']
assert client.get(MIGRATION_URL + '?state=bogus',
headers=auth_headers).status_code == 400
def test_every_blocker_is_listed(client, db, app, auth_headers, monkeypatch):
from plugins.endpoints import commands
from plugins.endpoints.api import enrollment
# No package in the store at all.
_pc(db, 'DUP')
_pc(db, 'dup', agentversion='1.5.0')
lockedid = _pc(db, 'LOCKED')
commands.PLACE_ENDPOINT_LOCK(assetid=lockedid, reason='bench repair')
db.session.commit()
monkeypatch.setattr(enrollment, 'minimum_agent_version_for',
lambda action: '1.10.0')
rows = {}
for row in _rows(client, auth_headers)['computers']:
rows.setdefault(row['hostname'], []).append(row)
assert rows['dup'][0]['blockers'] == ['duplicate hostname',
'no agent package in store',
'agent too old for remote upgrade']
assert rows['LOCKED'][0]['blockers'] == ['commands locked',
'no agent package in store']
def test_an_old_store_build_counts_as_no_package(client, db, app, auth_headers):
_publish_agent('1.19.5')
_pc(db, 'PC-A')
row = _by_host(_rows(client, auth_headers))['PC-A']
assert row['blockers'] == ['no agent package in store']
def test_version_compare_is_numeric(client, db, app, auth_headers):
_publish_agent('1.20.0')
_pc(db, 'PC-NINE', agentversion='1.9.0')
_pc(db, 'PC-HUNDRED', agentversion='1.100.0')
rows = _by_host(_rows(client, auth_headers))
assert rows['PC-NINE']['state'] == 'ready'
assert rows['PC-HUNDRED']['blockers'] == ['awaiting enrollment']
def test_wave_next_n_queues_ready_pcs_in_hostname_order(client, db, app,
auth_headers):
from plugins.endpoints.models import EndpointCommand, EndpointMigrationWave
_publish_agent()
third = _pc(db, 'PC-C')
first = _pc(db, 'PC-A')
second = _pc(db, 'PC-B')
_pc(db, 'PC-OFFLINE', reportedago=timedelta(days=2))
response = client.post(WAVE_URL, json={'next': 2}, headers=auth_headers)
assert response.status_code == 201, response.get_data(as_text=True)
data = response.get_json()['data']
assert data['queued'] == [first, second]
assert data['skipped'] == []
commands = EndpointCommand.query.filter_by(action='run_script').all()
assert sorted(c.assetid for c in commands) == [first, second]
wave = EndpointMigrationWave.query.one()
assert wave.waveid == data['waveid']
assert wave.assetidlist == [first, second]
migration = _rows(client, auth_headers)
rows = _by_host(migration)
assert rows['PC-A']['state'] == 'upgrading'
assert rows['PC-C']['state'] == 'ready'
assert migration['waves'] == [{
'waveid': wave.waveid, 'createdat': migration['waves'][0]['createdat'],
'createdby': 'testadmin', 'size': 2, 'queued': 2, 'enrolled': 0,
'failed': 0, 'done': False}]
assert third not in data['queued']
def test_wave_explicit_ids_rechecks_readiness(client, db, app, auth_headers):
_publish_agent()
ready = _pc(db, 'PC-READY')
offline = _pc(db, 'PC-OFFLINE', reportedago=timedelta(days=2))
noagent = _pc(db, 'PC-NOAGENT', reports=False)
response = client.post(WAVE_URL,
json={'assetids': [ready, offline, noagent, 999999]},
headers=auth_headers)
assert response.status_code == 201, response.get_data(as_text=True)
data = response.get_json()['data']
assert data['queued'] == [ready]
assert data['skipped'] == [
{'assetid': offline, 'reason': 'offline'},
{'assetid': noagent, 'reason': 'notreporting'},
{'assetid': 999999, 'reason': 'not a computer'},
]
def test_wave_packages_once(client, db, app, auth_headers, monkeypatch):
from plugins.endpoints.api import routes
_publish_agent()
ids = [_pc(db, f'PC-{n}') for n in range(3)]
calls = []
real = routes.prepare_agent_upgrade
def counting():
calls.append(1)
return real()
monkeypatch.setattr(routes, 'prepare_agent_upgrade', counting)
response = client.post(WAVE_URL, json={'assetids': ids},
headers=auth_headers)
assert response.status_code == 201
assert response.get_json()['data']['queued'] == ids
assert calls == [1]
def test_wave_progress_counts_enrolled_and_failed(client, db, app,
auth_headers):
from plugins.endpoints.api.enrollment import issue_credential
from plugins.endpoints.models import EndpointCommand
_publish_agent()
good = _pc(db, 'PC-GOOD')
bad = _pc(db, 'PC-BAD')
client.post(WAVE_URL, json={'assetids': [good, bad]}, headers=auth_headers)
issue_credential(good, 'auto')
EndpointCommand.query.filter_by(assetid=bad).one().status = \
EndpointCommand.STATUS_FAILED
db.session.commit()
wave = _rows(client, auth_headers)['waves'][0]
assert (wave['size'], wave['enrolled'], wave['failed'], wave['done']) == \
(2, 1, 1, True)
def test_wave_without_ready_pcs_creates_nothing(client, db, app, auth_headers):
from plugins.endpoints.models import EndpointMigrationWave
offline = _pc(db, 'PC-OFFLINE', reportedago=timedelta(days=2))
response = client.post(WAVE_URL, json={'assetids': [offline]},
headers=auth_headers)
assert response.status_code == 200
assert response.get_json()['data']['waveid'] is None
assert EndpointMigrationWave.query.count() == 0
@pytest.mark.parametrize('body', [{}, {'next': 0}, {'next': 101},
{'assetids': []}, {'assetids': ['1']},
{'assetids': list(range(1, 102))}])
def test_wave_validation(client, db, app, auth_headers, body):
assert client.post(WAVE_URL, json=body,
headers=auth_headers).status_code == 400

View File

@@ -1,254 +1,114 @@
"""Test fleet alert scheduling respects starttime.
"""Fleet alert scheduling respects starttime and endtime.
Regression test for bug where fleet alerts with future starttime were sent
immediately instead of waiting for the scheduled time.
Regression test for a bug where fleet alerts with a future starttime were sent
immediately instead of waiting for the scheduled time. The agent polls on the
collector credential, and the hostname must resolve to a Computer, so the
fixtures mirror test_endpoints_fleet_alerts.py.
"""
import pytest
from datetime import datetime, timedelta, timezone
from shopdb.core.models import Asset, AssetType
def test_fleet_alert_respects_future_starttime(client, auth_headers, sample_asset):
"""Fleet alerts with future starttime should not appear until that time."""
ALERTS_URL = '/api/endpoints/alerts'
COLLECTOR_KEY = 'testcollectorkey'
def _collector_headers(app):
app.config['COLLECTOR_API_KEY'] = COLLECTOR_KEY
return {'X-API-Key': COLLECTOR_KEY}
def _computer(db, hostname):
from plugins.computers.models import Computer
atype = AssetType.query.filter_by(assettype='computer').first()
if not atype:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
isactive=True)
db.session.add(asset)
db.session.flush()
db.session.add(Computer(assetid=asset.assetid, hostname=hostname))
db.session.commit()
return asset
def _fleet_alert(db, message, starttime=None, endtime=None):
from plugins.notifications.models import Notification, NotificationType
from plugins.endpoints.models import EndpointState
from shopdb.api import db
# Create notification type
ntype = NotificationType(
typename='System Maintenance',
typecolor='#ffc107',
isactive=True
)
db.session.add(ntype)
db.session.flush()
ntype = NotificationType.query.filter_by(typename='Fleet Alert').first()
if not ntype:
ntype = NotificationType(typename='Fleet Alert', typecolor='#ffc107',
isactive=True)
db.session.add(ntype)
db.session.flush()
note = Notification(notification=message,
notificationtypeid=ntype.notificationtypeid,
starttime=starttime, endtime=endtime, isactive=True,
alertfleet=True, alertscope=None, alerttimeout=0)
db.session.add(note)
db.session.commit()
return note
# Create endpoint state for the asset
state = EndpointState(
assetid=sample_asset.assetid,
hostname='TEST-PC-01',
agentversion='1.0.0',
lastreportedat=datetime.utcnow()
)
db.session.add(state)
db.session.flush()
now = datetime.now(timezone.utc).replace(tzinfo=None)
future_time = now + timedelta(minutes=20) # 20 minutes from now
end_time = future_time + timedelta(hours=1)
def _now():
return datetime.now(timezone.utc).replace(tzinfo=None)
# Create fleet alert scheduled for 20 minutes in the future
notification = Notification(
notification='Scheduled maintenance at 3:30 PM',
notificationtypeid=ntype.notificationtypeid,
starttime=future_time,
endtime=end_time,
isactive=True,
alertfleet=True,
alertscope=None, # All endpoints
alerttimeout=0
)
db.session.add(notification)
def _poll(client, app, hostname):
response = client.get(f'{ALERTS_URL}?hostname={hostname}',
headers=_collector_headers(app))
assert response.status_code == 200, response.get_data(as_text=True)
body = response.get_json()
assert body['status'] == 'success'
return body['data']
def test_fleet_alert_respects_future_starttime(client, db, app):
"""A future starttime keeps the alert off the endpoint until it passes."""
_computer(db, 'TEST-PC-01')
now = _now()
note = _fleet_alert(db, 'Scheduled maintenance at 3:30 PM',
starttime=now + timedelta(minutes=20),
endtime=now + timedelta(hours=2))
assert _poll(client, app, 'TEST-PC-01') == []
note.starttime = now - timedelta(minutes=5)
db.session.commit()
# Agent polls for fleet alerts NOW (before starttime)
response = client.get(
'/api/endpoints/alerts?hostname=TEST-PC-01',
headers=auth_headers
)
assert response.status_code == 200
data = response.get_json()
assert data['status'] == 'success'
# BUG FIX: Alert should NOT appear because starttime is in the future
alerts = data['data']
assert len(alerts) == 0, 'Fleet alert with future starttime should not appear yet'
# Simulate time passing - set starttime to the past
notification.starttime = now - timedelta(minutes=5)
db.session.commit()
# Agent polls again (now starttime has passed)
response = client.get(
'/api/endpoints/alerts?hostname=TEST-PC-01',
headers=auth_headers
)
assert response.status_code == 200
data = response.get_json()
assert data['status'] == 'success'
# Now the alert SHOULD appear
alerts = data['data']
assert len(alerts) == 1
assert alerts[0]['notificationId'] == notification.notificationid
alerts = _poll(client, app, 'TEST-PC-01')
assert [a['notificationId'] for a in alerts] == [note.notificationid]
assert alerts[0]['title'] == 'Scheduled maintenance at 3:30 PM'
def test_fleet_alert_with_no_starttime_appears_immediately(client, auth_headers, sample_asset):
"""Fleet alerts with no starttime should appear immediately."""
from plugins.notifications.models import Notification, NotificationType
from plugins.endpoints.models import EndpointState
from shopdb.api import db
def test_fleet_alert_with_no_starttime_appears_immediately(client, db, app):
_computer(db, 'TEST-PC-02')
note = _fleet_alert(db, 'Immediate alert - urgent')
# Create notification type
ntype = NotificationType(
typename='System Alert',
typecolor='#dc3545',
isactive=True
)
db.session.add(ntype)
db.session.flush()
# Create endpoint state
state = EndpointState(
assetid=sample_asset.assetid,
hostname='TEST-PC-02',
agentversion='1.0.0',
lastreportedat=datetime.utcnow()
)
db.session.add(state)
db.session.flush()
# Create fleet alert with no starttime (immediate)
notification = Notification(
notification='Immediate alert - urgent',
notificationtypeid=ntype.notificationtypeid,
starttime=None, # No starttime = immediate
endtime=None,
isactive=True,
alertfleet=True,
alertscope=None,
alerttimeout=0
)
db.session.add(notification)
db.session.commit()
# Agent polls for fleet alerts
response = client.get(
'/api/endpoints/alerts?hostname=TEST-PC-02',
headers=auth_headers
)
assert response.status_code == 200
data = response.get_json()
assert data['status'] == 'success'
# Alert should appear immediately
alerts = data['data']
assert len(alerts) == 1
assert alerts[0]['notificationId'] == notification.notificationid
alerts = _poll(client, app, 'TEST-PC-02')
assert [a['notificationId'] for a in alerts] == [note.notificationid]
assert alerts[0]['title'] == 'Immediate alert - urgent'
def test_fleet_alert_past_starttime_appears(client, auth_headers, sample_asset):
"""Fleet alerts with past starttime should appear."""
from plugins.notifications.models import Notification, NotificationType
from plugins.endpoints.models import EndpointState
from shopdb.api import db
def test_fleet_alert_past_starttime_appears(client, db, app):
_computer(db, 'TEST-PC-03')
note = _fleet_alert(db, 'Alert started 1 hour ago',
starttime=_now() - timedelta(hours=1))
ntype = NotificationType(
typename='Update',
typecolor='#17a2b8',
isactive=True
)
db.session.add(ntype)
db.session.flush()
state = EndpointState(
assetid=sample_asset.assetid,
hostname='TEST-PC-03',
agentversion='1.0.0',
lastreportedat=datetime.utcnow()
)
db.session.add(state)
db.session.flush()
now = datetime.now(timezone.utc).replace(tzinfo=None)
past_time = now - timedelta(hours=1)
# Create fleet alert that started 1 hour ago
notification = Notification(
notification='Alert started 1 hour ago',
notificationtypeid=ntype.notificationtypeid,
starttime=past_time,
endtime=None,
isactive=True,
alertfleet=True,
alertscope=None,
alerttimeout=0
)
db.session.add(notification)
db.session.commit()
# Agent polls for fleet alerts
response = client.get(
'/api/endpoints/alerts?hostname=TEST-PC-03',
headers=auth_headers
)
assert response.status_code == 200
data = response.get_json()
assert data['status'] == 'success'
# Alert should appear (starttime is in the past)
alerts = data['data']
assert len(alerts) == 1
assert alerts[0]['notificationId'] == notification.notificationid
alerts = _poll(client, app, 'TEST-PC-03')
assert [a['notificationId'] for a in alerts] == [note.notificationid]
def test_fleet_alert_expired_does_not_appear(client, auth_headers, sample_asset):
"""Fleet alerts with past endtime should not appear."""
from plugins.notifications.models import Notification, NotificationType
from plugins.endpoints.models import EndpointState
from shopdb.api import db
def test_fleet_alert_expired_does_not_appear(client, db, app):
_computer(db, 'TEST-PC-04')
now = _now()
_fleet_alert(db, 'This alert already ended',
starttime=now - timedelta(hours=2),
endtime=now - timedelta(hours=1))
ntype = NotificationType(
typename='Expired',
typecolor='#6c757d',
isactive=True
)
db.session.add(ntype)
db.session.flush()
state = EndpointState(
assetid=sample_asset.assetid,
hostname='TEST-PC-04',
agentversion='1.0.0',
lastreportedat=datetime.utcnow()
)
db.session.add(state)
db.session.flush()
now = datetime.now(timezone.utc).replace(tzinfo=None)
past_start = now - timedelta(hours=2)
past_end = now - timedelta(hours=1)
# Create expired fleet alert
notification = Notification(
notification='This alert already ended',
notificationtypeid=ntype.notificationtypeid,
starttime=past_start,
endtime=past_end, # Ended 1 hour ago
isactive=True,
alertfleet=True,
alertscope=None,
alerttimeout=0
)
db.session.add(notification)
db.session.commit()
# Agent polls for fleet alerts
response = client.get(
'/api/endpoints/alerts?hostname=TEST-PC-04',
headers=auth_headers
)
assert response.status_code == 200
data = response.get_json()
assert data['status'] == 'success'
# Alert should NOT appear (endtime is in the past)
alerts = data['data']
assert len(alerts) == 0
assert _poll(client, app, 'TEST-PC-04') == []

View File

@@ -427,14 +427,42 @@ def test_an_unknown_scope_key_is_ignored_not_fatal(client, db, app):
assert _ids(client, app, 'BAY-01') == [alert.notificationid]
def test_a_non_dict_scope_falls_back_to_the_whole_fleet(client, db, app):
"""Garbled scope JSON must not stop an alert reaching the floor."""
def test_a_non_dict_scope_fails_closed(client, db, app):
"""A scope that is present but not an object is corrupt, not fleet-wide.
Create/update refuse such a scope, so a row carrying one did not come from
an operator's intent to reach everyone. Broadcasting it could put a
one-bay alert on every screen, so it reaches nobody.
"""
_computer(db, 'BAY-01')
alert = _fleet_alert(db)
alert.alertscope = 'BAY-01'
db.session.commit()
assert _ids(client, app, 'BAY-01') == []
alert.alertscope = ['BAY-01']
db.session.commit()
assert _ids(client, app, 'BAY-01') == []
def test_a_json_string_scope_is_parsed(client, db, app):
"""Legacy rows written through the TEXT column hold the object as text."""
_computer(db, 'BAY-01')
_computer(db, 'BAY-02')
alert = _fleet_alert(db)
alert.alertscope = '{"hostname": "BAY-01"}'
db.session.commit()
assert _ids(client, app, 'BAY-01') == [alert.notificationid]
assert _ids(client, app, 'BAY-02') == []
def test_scope_keys_match_the_notifications_validator():
"""Create/update validation and the matcher must agree on the key set."""
from plugins.endpoints.api.routes import _ALERT_SCOPE_KEYS
from plugins.notifications.models.notification import ALERT_SCOPE_KEYS
assert set(_ALERT_SCOPE_KEYS) == set(ALERT_SCOPE_KEYS)
def test_scoping_does_not_leak_into_acknowledgement(client, db, app):

View File

@@ -0,0 +1,53 @@
"""Health evaluation failing with a database error must not sink the check-in.
The health rollup is fail-open: a caught exception leaves the previous verdict
standing. A caught IntegrityError from a flush, though, leaves the session
needing a rollback, and the outer commit of the whole check-in would then fail.
The evaluation runs inside a SAVEPOINT so only its own writes are discarded.
"""
from shopdb.core.models import Asset, AssetType
def _computer(db, hostname):
from plugins.computers.models import Computer
atype = AssetType.query.filter_by(assettype='computer').first()
if not atype:
atype = AssetType(assettype='computer')
db.session.add(atype)
db.session.flush()
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
isactive=True)
db.session.add(asset)
db.session.flush()
db.session.add(Computer(assetid=asset.assetid, hostname=hostname))
db.session.commit()
return asset
def test_integrity_error_in_health_keeps_the_checkin(client, db, app,
monkeypatch):
from plugins.endpoints.plugin import EndpointsPlugin
from plugins.endpoints.models import EndpointState
from plugins.endpoints.services import health
calls = []
def exploding_health(assetid, state, computertypeid=None):
calls.append(assetid)
# A duplicate unique value: the flush raises IntegrityError.
db.session.add(AssetType(assettype='computer'))
db.session.flush()
monkeypatch.setattr(health, 'evaluate_endpoint_health', exploding_health)
asset = _computer(db, 'SAVEPT01')
plugin = EndpointsPlugin()
plugin.apply_collector_payload({'hostname': 'SAVEPT01', 'services': []})
db.session.commit()
assert calls == [asset.assetid]
state = EndpointState.query.filter_by(assetid=asset.assetid).first()
assert state is not None
assert AssetType.query.filter_by(assettype='computer').count() == 1

View File

@@ -175,6 +175,44 @@ def test_an_admin_override_is_accepted_and_audited(client, db):
assert entry is not None, 'break-glass must leave an audit record'
def test_a_refused_override_leaves_no_hold_and_no_audit_row(client, db):
"""Validation after the override branch must run before anything is written.
A targetless service action, and a second command while one is in flight,
are both refused. Neither may leave a policyoverridden audit row or a
PolicyActionHold behind, or the record claims a break-glass that never ran.
"""
from shopdb.api import AuditLog
from plugins.computers.models import PolicyActionHold
from plugins.endpoints.models import EndpointCommand
_enable_policies(db)
assetid = _endpoint(db, 'refusedbox')
_deny_everything(db, assetid)
headers = _user(db, client, 'refusedadmin',
['endpoints.view', 'endpoints.command'], admin=True)
missing_target = client.post(_url(assetid), headers=headers,
json={'action': 'service_status',
'policyoverride': True,
'overridereason': 'no target given'})
assert missing_target.status_code == 400, missing_target.get_json()
db.session.add(EndpointCommand(assetid=assetid, action='collect_now',
status=EndpointCommand.STATUS_PENDING))
db.session.commit()
in_flight = client.post(_url(assetid), headers=headers,
json={'action': 'service_status',
'target': 'spooler',
'policyoverride': True,
'overridereason': 'queue is busy'})
assert in_flight.status_code == 409, in_flight.get_json()
db.session.expire_all()
assert AuditLog.query.filter_by(action='policyoverridden').count() == 0
assert PolicyActionHold.query.filter_by(assetid=assetid).count() == 0
def test_override_is_checked_after_the_elevated_permission_gate(client, db):
"""The override widens WHERE an action may run, never WHICH actions the
caller may use.

View File

@@ -0,0 +1,157 @@
"""Server TLS certificate upload and generate routes.
The new pair must be parsed, and the key proven to match the certificate,
BEFORE the existing files are backed up or replaced: a bad upload must leave
nginx's current certificate exactly where it was. cn and san must be plain
hostname/IP syntax; anything else (newlines, '=', brackets) is refused.
"""
import io
from datetime import datetime, timedelta, timezone
import pytest
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.x509.oid import NameOID
from plugins.endpoints.api import routes as endpoint_routes
UPLOAD_URL = '/api/endpoints/settings/server-cert/upload'
GENERATE_URL = '/api/endpoints/settings/server-cert/generate'
@pytest.fixture(autouse=True)
def cert_dir(tmp_path, monkeypatch):
directory = tmp_path / 'certs'
directory.mkdir()
monkeypatch.setattr(endpoint_routes, '_server_cert_dir', lambda: directory)
monkeypatch.setattr(endpoint_routes, '_restart_nginx', lambda: None)
return directory
def _pair(cn='old.example.test'):
key = ec.generate_private_key(ec.SECP256R1())
name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, cn)])
now = datetime.now(timezone.utc)
certificate = (x509.CertificateBuilder()
.subject_name(name).issuer_name(name)
.public_key(key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now - timedelta(days=1))
.not_valid_after(now + timedelta(days=30))
.sign(key, hashes.SHA256()))
cert_pem = certificate.public_bytes(serialization.Encoding.PEM)
key_pem = key.private_bytes(serialization.Encoding.PEM,
serialization.PrivateFormat.PKCS8,
serialization.NoEncryption())
return cert_pem, key_pem
def _seed_existing(cert_dir):
cert_pem, key_pem = _pair('existing.example.test')
(cert_dir / 'server.crt').write_bytes(cert_pem)
(cert_dir / 'server.key').write_bytes(key_pem)
return cert_pem, key_pem
def _upload(client, headers, cert_bytes, key_bytes):
return client.post(UPLOAD_URL, headers=headers,
content_type='multipart/form-data',
data={'cert': (io.BytesIO(cert_bytes), 'server.crt'),
'key': (io.BytesIO(key_bytes), 'server.key')})
def test_a_matching_pair_replaces_and_backs_up(client, db, auth_headers, cert_dir):
from shopdb.api import AuditLog
old_cert, old_key = _seed_existing(cert_dir)
new_cert, new_key = _pair('new.example.test')
response = _upload(client, auth_headers, new_cert, new_key)
assert response.status_code == 200, response.get_json()
assert (cert_dir / 'server.crt').read_bytes() == new_cert
assert (cert_dir / 'server.key').read_bytes() == new_key
backups = sorted(p.name for p in cert_dir.iterdir() if '.backup.' in p.name)
assert len(backups) == 2
assert not [p for p in cert_dir.iterdir() if p.name.endswith('.tmp')]
assert AuditLog.query.filter_by(
action='endpoints.server_cert.uploaded').count() == 1
def test_a_mismatched_key_touches_nothing(client, db, auth_headers, cert_dir):
old_cert, old_key = _seed_existing(cert_dir)
new_cert, _new_key = _pair('new.example.test')
_other_cert, other_key = _pair('other.example.test')
response = _upload(client, auth_headers, new_cert, other_key)
assert response.status_code == 400
assert 'does not match' in response.get_json()['data']['error']['message']
assert (cert_dir / 'server.crt').read_bytes() == old_cert
assert (cert_dir / 'server.key').read_bytes() == old_key
assert sorted(p.name for p in cert_dir.iterdir()) == ['server.crt', 'server.key']
def test_garbage_certificate_touches_nothing(client, db, auth_headers, cert_dir):
old_cert, _old_key = _seed_existing(cert_dir)
_cert, key = _pair()
garbage = b'-----BEGIN CERTIFICATE-----\nnope\n-----END CERTIFICATE-----\n'
response = _upload(client, auth_headers, garbage, key)
assert response.status_code == 400
assert (cert_dir / 'server.crt').read_bytes() == old_cert
assert len(list(cert_dir.iterdir())) == 2
@pytest.mark.parametrize('cn', [
'host\nprompt = yes',
'bad host',
'a=b',
'[alt_names]',
'a' * 65,
])
def test_generate_refuses_unsafe_cn(client, db, auth_headers, cert_dir, cn):
_seed_existing(cert_dir)
response = client.post(GENERATE_URL, headers=auth_headers,
json={'cn': cn, 'days': 30})
assert response.status_code == 400
assert len(list(cert_dir.iterdir())) == 2
@pytest.mark.parametrize('san', [
'DNS:ok.example.test,DNS:bad\nname',
'IP:999.1.1.1',
'URI:http://x',
'DNS:a b',
])
def test_generate_refuses_unsafe_san(client, db, auth_headers, cert_dir, san):
_seed_existing(cert_dir)
response = client.post(GENERATE_URL, headers=auth_headers,
json={'cn': 'ok.example.test', 'san': san,
'days': 30})
assert response.status_code == 400
assert len(list(cert_dir.iterdir())) == 2
def test_generate_installs_a_valid_pair(client, db, auth_headers, cert_dir):
from shopdb.api import AuditLog
_seed_existing(cert_dir)
response = client.post(GENERATE_URL, headers=auth_headers,
json={'cn': 'shopdb.example.test',
'san': 'DNS:shopdb.example.test,IP:10.0.0.5',
'days': 30})
assert response.status_code == 200, response.get_json()
certificate = x509.load_pem_x509_certificate(
(cert_dir / 'server.crt').read_bytes())
assert certificate.subject.get_attributes_for_oid(
NameOID.COMMON_NAME)[0].value == 'shopdb.example.test'
assert endpoint_routes._validate_certificate_pair(
(cert_dir / 'server.crt').read_bytes(),
(cert_dir / 'server.key').read_bytes()) is None
assert AuditLog.query.filter_by(
action='endpoints.server_cert.generated').count() == 1

View File

@@ -0,0 +1,101 @@
"""Fleet alert fields on notification create/update.
alertfleet and alerttimeout are NOT NULL in migration 0006, so a null from the
form is coerced to the column default rather than reaching the database.
alertscope must be an object whose keys the endpoint matcher understands; a
typo or a non-object is refused at the API instead of being stored and
silently re-interpreted at delivery time.
"""
from plugins.notifications.models import Notification, NotificationType
def _type(db):
ntype = NotificationType(typename='Fleet Alert', typecolor='#d93f3f',
isactive=True)
db.session.add(ntype)
db.session.commit()
return ntype.notificationtypeid
def _create(client, headers, typeid, **fields):
body = {'notification': 'Line 3 is down', 'notificationtypeid': typeid}
body.update(fields)
return client.post('/api/notifications', json=body, headers=headers)
def test_null_timeout_and_fleet_are_coerced_to_defaults(client, db, auth_headers):
typeid = _type(db)
resp = _create(client, auth_headers, typeid, alertfleet=None,
alerttimeout=None, alertscope=None)
assert resp.status_code == 201, resp.get_json()
row = db.session.get(Notification, resp.get_json()['data']['notificationid'])
assert row.alertfleet is False
assert row.alerttimeout == 0
assert row.alertscope is None
def test_update_coerces_null_timeout(client, db, auth_headers):
typeid = _type(db)
created = _create(client, auth_headers, typeid, alertfleet=True,
alerttimeout=60).get_json()['data']
resp = client.put(f"/api/notifications/{created['notificationid']}",
json={'alerttimeout': None, 'alertfleet': None},
headers=auth_headers)
assert resp.status_code == 200, resp.get_json()
row = db.session.get(Notification, created['notificationid'])
assert row.alerttimeout == 0
assert row.alertfleet is False
def test_a_valid_scope_is_stored(client, db, auth_headers):
typeid = _type(db)
resp = _create(client, auth_headers, typeid, alertfleet=True,
alertscope={'hostname': ['BAY-01', 'BAY-02'],
'computertype': 'CMM PC'})
assert resp.status_code == 201, resp.get_json()
assert resp.get_json()['data']['alertscope']['computertype'] == 'CMM PC'
def test_an_empty_scope_means_whole_fleet(client, db, auth_headers):
typeid = _type(db)
resp = _create(client, auth_headers, typeid, alertfleet=True, alertscope='')
assert resp.status_code == 201, resp.get_json()
assert resp.get_json()['data']['alertscope'] is None
def test_unknown_scope_key_is_refused(client, db, auth_headers):
typeid = _type(db)
resp = _create(client, auth_headers, typeid, alertfleet=True,
alertscope={'hostnam': 'BAY-01'})
assert resp.status_code == 400
assert 'hostnam' in resp.get_json()['data']['error']['message']
assert Notification.query.count() == 0
def test_non_object_scope_is_refused(client, db, auth_headers):
typeid = _type(db)
for scope in (['BAY-01'], 'BAY-01', '{not json', 5):
resp = _create(client, auth_headers, typeid, alertfleet=True,
alertscope=scope)
assert resp.status_code == 400, scope
assert Notification.query.count() == 0
def test_update_refuses_bad_scope_and_keeps_the_old_one(client, db, auth_headers):
typeid = _type(db)
created = _create(client, auth_headers, typeid, alertfleet=True,
alertscope={'hostname': 'BAY-01'}).get_json()['data']
resp = client.put(f"/api/notifications/{created['notificationid']}",
json={'alertscope': {'building': 'X'}},
headers=auth_headers)
assert resp.status_code == 400
db.session.expire_all()
row = db.session.get(Notification, created['notificationid'])
assert row.alertscope == {'hostname': 'BAY-01'}
def test_bad_timeout_is_refused(client, db, auth_headers):
typeid = _type(db)
assert _create(client, auth_headers, typeid, alerttimeout=-1).status_code == 400
assert _create(client, auth_headers, typeid, alerttimeout='soon').status_code == 400