Give each endpoint its own credential and fix the review findings on the server

Per-PC credentials (replaces the 09-14 client-certificate attempt):
- POST /api/endpoints/enroll issues a credential to a PC that matches exactly
  one computer by hostname and BIOS serial and has none yet. Anything else
  (unknown or duplicate hostname, serial mismatch, already enrolled) waits as
  a request an admin approves or rejects. The agent's persisted nonce binds an
  approved request to the PC that asked.
- One auth helper serves every agent route. With X-Endpoint-Credential the
  asset comes from the credential and a request naming another PC gets 403
  ASSET_MISMATCH. The shared key still works for PCs that have not enrolled,
  is refused (401 SHARED_KEY_REFUSED) for any PC that has, and can be switched
  off. A revoked credential gets 401 CREDENTIAL_INVALID so the agent
  re-enrolls.
- The core collector accepts the credential for the computers, printers and
  endpoints collectors through two new plugin hooks exposed on shopdb.api
  (contract 0.33.0), and refuses the shared key for enrolled PCs.
- Admin API: enrollment list, approve, reject, revoke (un-enroll), settings.
  Migration API: per-PC readiness with blockers, and upgrade waves that
  package the agent once and queue the existing agent upgrade per PC.
- Migration 0023 creates endpointcredentials, endpointenrollrequests and
  endpointmigrationwaves, drops the certificate columns from assets where
  test databases have them, and renames app_fingerprint to appfingerprint.
  0021 and 0022 keep their ids but no longer alter the core assets table.
- Removed: client_cert.py, certificate renewal, the unauthenticated
  /test/client-cert route, nginx client certificate forwarding and the agent
  certificate script. The client-cert header could be forged on IIS, the key
  was readable by users, and the shared key remained a fallback.
- GET /settings/server-cert returns the SPKI SHA-256 pin for GE-Enforce.

Review fixes:
- report_command_result used g without importing it; every result was a 500.
- collect_log refuses setup answer files, appsettings, agent directories and
  key files at enqueue.
- Fleet alerts: alerttimeout and alertfleet agree with the migration, the
  scope is validated and a malformed scope no longer reaches the whole fleet,
  and the ack race no longer raises NameError.
- A refused break-glass override leaves no hold and no audit row.
- Agent upgrade checks locks and permissions before repackaging, and writes
  the package atomically.
- Server certificate upload validates the pair before replacing files;
  generation no longer builds an openssl config from user input; audit rows
  are actually written.
- Health evaluation runs in a savepoint so a caught error cannot lose the
  check-in.
- Required-app policies are found by appid, survive renames, and deleting an
  app deactivates policies that still name it. The UI and API now say such a
  policy only takes effect inside a business policy.
- Backfill handles duplicate fingerprints and rolls back a failed batch.
- Plugin settings routes skip a plugin whose hook raises.
- The setup token is no longer printed to the log.
- The naming gate checks top-level files again and handles odd file names.
- Removed root scratch scripts, one of which carried a token.
- Rewrote the fleet alert scheduling and app delta tests on real fixtures.
This commit is contained in:
cproudlock
2026-09-15 12:03:53 -04:00
parent f484be8a33
commit 7480e4fe3a
58 changed files with 4485 additions and 2032 deletions

View File

@@ -9,5 +9,7 @@ from . import siteagent # noqa: F401
from . import agentupload # noqa: F401
# Attaches the monitor catalog CRUD and resolve routes to endpoints_bp.
from . import monitors # noqa: F401
# Attaches per-PC credential enrollment, admin and migration routes.
from . import enrollment # noqa: F401
__all__ = ['endpoints_bp']

View File

@@ -0,0 +1,196 @@
"""Agent authentication: per-PC credentials, with the shared key as fallback.
Every agent-facing endpoints route authenticates through authenticate_agent()
and, when the route names a PC, binds the caller through bind_hostname().
X-Endpoint-Credential The PC's own secret. The server derives the asset from
it; a hostname or assetid in the request that names a
different asset is refused with ASSET_MISMATCH.
X-API-Key The shared fleet key (or a collector-scoped managed
token). Accepted only while
plugin.endpoints.sharedkeyallowed is on, and never for a
PC that holds an active credential.
Only the sha256 of a credential is stored. The full secret exists once, in the
enroll response.
"""
import hashlib
import hmac
import re
import secrets
from datetime import datetime, timedelta, timezone
from flask import g, request
from shopdb.api import (
db, Setting, error_response, ErrorCodes, collector_authorized,
)
CREDENTIAL_HEADER = 'X-Endpoint-Credential'
CREDENTIAL_PREFIX = 'sdbe_'
# sdbe_ + 43 chars base64url (32 random bytes, unpadded).
_CREDENTIAL_PATTERN = re.compile(r'^sdbe_[A-Za-z0-9_-]{43}$')
DISPLAY_PREFIX_LENGTH = 12
SHARED_KEY_SETTING = 'plugin.endpoints.sharedkeyallowed'
AUTO_ENROLL_SETTING = 'plugin.endpoints.autoenroll'
CODE_CREDENTIAL_INVALID = 'CREDENTIAL_INVALID'
CODE_SHARED_KEY_REFUSED = 'SHARED_KEY_REFUSED'
CODE_ASSET_MISMATCH = 'ASSET_MISMATCH'
AUTH_CREDENTIAL = 'credential'
AUTH_SHARED_KEY = 'sharedkey'
# lastusedat is a display value, not an audit trail: one write per 5 minutes
# per PC keeps the long-poll from turning every claim into an UPDATE.
_TOUCH_INTERVAL = timedelta(minutes=5)
def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def generate_credential():
return CREDENTIAL_PREFIX + secrets.token_urlsafe(32)
def hash_credential(secret):
return hashlib.sha256(secret.encode('utf-8')).hexdigest()
def shared_key_allowed():
return Setting.get(SHARED_KEY_SETTING, default=True) is not False
def auto_enroll_enabled():
return Setting.get(AUTO_ENROLL_SETTING, default=True) is not False
def active_credential_for(assetid):
from ..models import EndpointCredential
if assetid is None:
return None
return (EndpointCredential.query
.filter(EndpointCredential.assetid == assetid,
EndpointCredential.revokedat.is_(None))
.first())
def resolve_credential(secret, touch=True):
"""Asset id for an active credential, else None."""
from ..models import EndpointCredential
if not secret or not _CREDENTIAL_PATTERN.match(secret):
return None
digest = hash_credential(secret)
row = EndpointCredential.query.filter_by(credentialhash=digest).first()
if row is None or row.revokedat is not None:
return None
if not hmac.compare_digest(row.credentialhash, digest):
return None
if touch:
now = _naive_utcnow()
if row.lastusedat is None or now - row.lastusedat >= _TOUCH_INTERVAL:
row.lastusedat = now
db.session.commit()
return row.assetid
def resolve_hostname(hostname):
"""Asset id of the one computer with this hostname (case-insensitive)."""
from plugins.computers.models import Computer
hostname = (hostname or '').strip()
if not hostname:
return None
computer = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.first())
return computer.assetid if computer is not None else None
def shared_key_refused_for(hostname):
"""True when the shared key may not report for this hostname."""
if not shared_key_allowed():
return True
assetid = resolve_hostname(hostname)
return assetid is not None and active_credential_for(assetid) is not None
def credential_invalid_response():
return error_response(
CODE_CREDENTIAL_INVALID,
'Endpoint credential is unknown or revoked; enroll again',
http_code=401)
def shared_key_refused_response():
return error_response(
CODE_SHARED_KEY_REFUSED,
'The shared key is not accepted for this endpoint; use its credential',
http_code=401)
def asset_mismatch_response():
return error_response(
CODE_ASSET_MISMATCH,
'The request names a different endpoint than the credential',
http_code=403)
def authenticate_agent():
"""Authorize an agent request. Returns None when authorized, else a response.
Sets g.agentauth ('credential' or 'sharedkey') and g.agentassetid (the
credential's asset, or None until bind_hostname resolves one).
"""
g.agentauth = None
g.agentassetid = None
secret = request.headers.get(CREDENTIAL_HEADER)
if secret:
assetid = resolve_credential(secret.strip())
if assetid is None:
return credential_invalid_response()
g.agentauth = AUTH_CREDENTIAL
g.agentassetid = assetid
return None
error = collector_authorized('endpoints')
if error is not None:
return error
if not shared_key_allowed():
return shared_key_refused_response()
g.agentauth = AUTH_SHARED_KEY
return None
def is_credential_caller():
return getattr(g, 'agentauth', None) == AUTH_CREDENTIAL
def bind_hostname(hostname):
"""The asset this request acts for. Returns (assetid, error_response).
Credential: the credential's asset; a hostname naming another asset is an
ASSET_MISMATCH, and a missing hostname is fine.
Shared key: the hostname's asset (None when absent or unknown, for the
route to answer); an enrolled PC is refused.
"""
hostname = (hostname or '').strip()
if is_credential_caller():
if hostname and resolve_hostname(hostname) != g.agentassetid:
return None, asset_mismatch_response()
return g.agentassetid, None
if not hostname:
return None, None
assetid = resolve_hostname(hostname)
if assetid is not None and active_credential_for(assetid) is not None:
return None, shared_key_refused_response()
g.agentassetid = assetid
return assetid, None

View File

@@ -14,6 +14,7 @@ hand-maintained; an operator never edits these rows.
"""
import os
import uuid
from shopdb.api import Application, application_package_location, db
@@ -91,11 +92,19 @@ def register_current_agent_package():
os.makedirs(os.path.dirname(path), exist_ok=True)
# Write to a temp name and replace, so an interrupted write cannot leave a
# truncated archive at the path the row already advertises. os.replace is
# atomic within a filesystem.
temppath = path + '.tmp'
with open(temppath, 'wb') as handle:
handle.write(payload)
os.replace(temppath, path)
# atomic within a filesystem. The temp name is unique per call so two
# concurrent pushes (a batch wave) cannot interleave writes into one file.
temppath = '{}.{}.tmp'.format(path, uuid.uuid4().hex)
try:
with open(temppath, 'wb') as handle:
handle.write(payload)
os.replace(temppath, path)
except BaseException:
try:
os.remove(temppath)
except OSError:
pass
raise
_directory, urlprefix = application_package_location()
app.installpath = '{}{}/{}'.format(

View File

@@ -23,6 +23,7 @@ substitution done after hashing would fail the agent's verification.
import hashlib
import os
import uuid
from shopdb.api import Script, db, script_body_location
@@ -115,10 +116,17 @@ def seed_upgrade_script():
os.makedirs(_script_dir(), exist_ok=True)
# Temp-then-replace so an interrupted write cannot leave a truncated
# body at the path the row's digest already claims to describe.
temppath = path + '.tmp'
with open(temppath, 'wb') as handle:
handle.write(body)
os.replace(temppath, path)
temppath = '{}.{}.tmp'.format(path, uuid.uuid4().hex)
try:
with open(temppath, 'wb') as handle:
handle.write(body)
os.replace(temppath, path)
except BaseException:
try:
os.remove(temppath)
except OSError:
pass
raise
script.filename = filename
script.sha256 = digest

View File

@@ -1,103 +0,0 @@
"""Server-side certificate generation for agent self-renewal.
Agents check certificate expiry on startup and request renewal when < 30 days
remaining. Server generates new self-signed certificate, computes fingerprint,
updates database, and returns certificate as password-protected PFX to agent.
"""
import hashlib
import secrets
from datetime import datetime, timedelta, timezone
from flask import current_app
from cryptography import x509
from cryptography.x509.oid import NameOID, ExtendedKeyUsageOID
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.backends import default_backend
def generate_client_certificate(hostname, validity_days=730):
"""Generate self-signed client certificate for endpoint agent.
Args:
hostname: Computer hostname (used as CN)
validity_days: Certificate validity period (default 730 = 2 years)
Returns:
tuple: (pfx_bytes, pfx_password, sha1_fingerprint, expires_at)
- pfx_bytes: Password-protected PFX/P12 certificate with private key
- pfx_password: Random password for PFX (agent discards after import)
- sha1_fingerprint: 40-character lowercase hex SHA1 fingerprint
- expires_at: Certificate expiration datetime (UTC)
"""
# Generate RSA private key (2048-bit for compatibility with Windows)
private_key = rsa.generate_private_key(
public_exponent=65537,
key_size=2048,
backend=default_backend()
)
# Certificate subject and issuer (self-signed, so both are the same)
subject = issuer = x509.Name([
x509.NameAttribute(NameOID.COMMON_NAME, hostname),
x509.NameAttribute(NameOID.ORGANIZATION_NAME, 'ShopDB Endpoint Fleet'),
x509.NameAttribute(NameOID.ORGANIZATIONAL_UNIT_NAME, 'Endpoint Agents'),
])
# Certificate validity period
now = datetime.now(timezone.utc)
not_valid_before = now
not_valid_after = now + timedelta(days=validity_days)
# Build certificate
cert = x509.CertificateBuilder().subject_name(
subject
).issuer_name(
issuer
).public_key(
private_key.public_key()
).serial_number(
x509.random_serial_number()
).not_valid_before(
not_valid_before
).not_valid_after(
not_valid_after
).add_extension(
# Client authentication extended key usage
x509.ExtendedKeyUsage([ExtendedKeyUsageOID.CLIENT_AUTH]),
critical=True,
).add_extension(
# Subject Alternative Name (hostname as DNS name)
x509.SubjectAlternativeName([x509.DNSName(hostname)]),
critical=False,
).add_extension(
# Basic constraints: not a CA
x509.BasicConstraints(ca=False, path_length=None),
critical=True,
).sign(private_key, hashes.SHA256(), default_backend())
# Compute SHA1 fingerprint (matches nginx $ssl_client_fingerprint)
cert_der = cert.public_bytes(serialization.Encoding.DER)
sha1_hash = hashlib.sha1(cert_der).hexdigest()
# Generate random PFX password (agent uses once during import, then discards)
pfx_password = secrets.token_urlsafe(32)
# Serialize to PFX/P12 format with password protection
# Windows Certificate Store imports PFX format
pfx_bytes = serialization.pkcs12.serialize_key_and_certificates(
name=hostname.encode('utf-8'),
key=private_key,
cert=cert,
cas=None, # No CA chain for self-signed
encryption_algorithm=serialization.BestAvailableEncryption(
pfx_password.encode('utf-8')
)
)
current_app.logger.info(
f'[cert_renewal] Generated certificate for {hostname}: '
f'fingerprint={sha1_hash[:16]}..., expires={not_valid_after}')
return (pfx_bytes, pfx_password, sha1_hash, not_valid_after.replace(tzinfo=None))

View File

@@ -0,0 +1,789 @@
"""Per-PC endpoint credentials: enrollment, administration and fleet migration.
Agent-facing:
POST /enroll issue a credential or record a pending request
POST /collect the generic collector ingest, under this prefix
Admin (endpoints.fleet to read, admin role to change):
GET /enrollment credential state per computer + requests
POST /enrollment/requests/<id>/approve | reject
POST /enrollment/assets/<assetid>/revoke
PUT /enrollment/settings
GET /enrollment/migration readiness of every computer for agent 1.20
POST /enrollment/migration/wave push the agent upgrade to a batch of PCs
A PC moves onto its own credential by running an agent that enrolls. The
migration surface computes, per computer, whether the existing agent upgrade
can be pushed to it, and pushes it in waves through the same enqueue path the
single-asset agentupgrade route uses.
"""
import hmac
import re
from collections import Counter
from datetime import datetime, timedelta, timezone
from flask import request
from flask_jwt_extended import current_user
from shopdb.api import (
db, Asset, AuditLog, Script, error_response, success_response, ErrorCodes,
require_permission, require_role, collector_authorized, collector_ingest,
client_ip, get_pagination_params,
)
from ..commands import (
ACTIVE_LOCK_FOR, ALLOWED_ACTIONS_FOR, agent_meets_minimum,
minimum_agent_version_for, parse_agent_version,
)
from ..models import (
EndpointCommand, EndpointCredential, EndpointEnrollRequest,
EndpointMigrationWave, EndpointState,
)
from . import agentauth
from .routes import endpoints_bp
# First agent that enrolls itself.
CREDENTIAL_AGENT_VERSION = '1.20.0'
CODE_ENROLL_NONCE_MISMATCH = 'ENROLL_NONCE_MISMATCH'
_NONCE_PATTERN = re.compile(r'^[A-Za-z0-9_-]{32,128}$')
REASON_UNKNOWN_HOSTNAME = 'unknown hostname'
REASON_DUPLICATE_HOSTNAME = 'duplicate hostname'
REASON_NO_SERIAL = 'no serial on record'
REASON_SERIAL_MISMATCH = 'serial mismatch'
REASON_ALREADY_ENROLLED = 'already enrolled'
REASON_AWAITING_APPROVAL = 'awaiting approval'
REASON_NONCE_CHANGED = 'nonce changed'
REASON_REJECTED = 'rejected'
MIGRATION_STATES = ('enrolled', 'pending', 'upgrading', 'ready', 'blocked',
'notreporting')
ENROLLMENT_STATES = ('enrolled', 'notenrolled', 'pending', 'revoked')
BLOCKER_OFFLINE = 'offline'
BLOCKER_NO_SERIAL = 'no serial on record'
BLOCKER_DUPLICATE_HOSTNAME = 'duplicate hostname'
BLOCKER_LOCKED = 'commands locked'
BLOCKER_NO_PACKAGE = 'no agent package in store'
BLOCKER_TOO_OLD = 'agent too old for remote upgrade'
BLOCKER_POLICY = 'run_script not permitted by policy'
BLOCKER_AWAITING_ENROLLMENT = 'awaiting enrollment'
_OFFLINE_AFTER = timedelta(hours=24)
_WAVE_MAXIMUM = 100
_RECENT_REQUESTS = 50
def _naive_utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def _utciso(value):
return value.isoformat() + 'Z' if value else None
def _username():
return getattr(current_user, 'username', None)
def _computer_model():
from plugins.computers.models import Computer
return Computer
# ---------------------------------------------------------------------------
# Agent-facing
# ---------------------------------------------------------------------------
@endpoints_bp.route('/collect', methods=['POST'])
def agent_collect():
"""Agent telemetry ingest. Same handler as POST /api/collector/endpoints."""
return collector_ingest('endpoints')
def _pending(row, reason, matchedassetid=None):
row.status = EndpointEnrollRequest.STATUS_PENDING
row.reason = reason
row.matchedassetid = matchedassetid
db.session.commit()
return success_response(
{'status': 'pending', 'requestid': row.requestid, 'reason': reason},
http_code=202)
def issue_credential(assetid, enrolledvia):
"""Issue (or re-issue after revoke) the credential for one asset.
Returns the secret. Only its hash is stored. Caller commits.
"""
secret = agentauth.generate_credential()
row = EndpointCredential.query.filter_by(assetid=assetid).first()
if row is None:
row = EndpointCredential(assetid=assetid)
db.session.add(row)
row.credentialhash = agentauth.hash_credential(secret)
row.credentialprefix = secret[:agentauth.DISPLAY_PREFIX_LENGTH]
row.enrolledvia = enrolledvia
row.issuedat = _naive_utcnow()
row.lastusedat = None
row.revokedat = None
row.revokedby = None
return secret, row
def _issue(row, assetid, enrolledvia):
secret, credential = issue_credential(assetid, enrolledvia)
row.status = EndpointEnrollRequest.STATUS_ISSUED
row.matchedassetid = assetid
row.reason = None
AuditLog.log('created', 'EndpointCredential', entityid=assetid,
entityname=row.hostname,
details='endpoint credential {} issued ({}), request {}'.format(
credential.credentialprefix, enrolledvia, row.requestid))
db.session.commit()
return success_response(
{'status': 'enrolled', 'credential': secret, 'assetid': assetid},
message='Endpoint enrolled', http_code=201)
def _text_field(payload, name, maximum, required):
value = payload.get(name)
if value is None:
value = ''
if not isinstance(value, str):
return None, f'{name} must be a string'
value = value.strip()
if required and not value:
return None, f'{name} is required'
if len(value) > maximum:
return None, f'{name} must be at most {maximum} characters'
return value, None
@endpoints_bp.route('/enroll', methods=['POST'])
def enroll_endpoint():
"""A PC asks for its own credential.
Shared-key authenticated while plugin.endpoints.sharedkeyallowed is on.
With it off the route takes no auth and only hands out a credential an
admin already approved for this exact hostname, serial and nonce.
"""
sharedkeyauthenticated = False
if agentauth.shared_key_allowed():
error = collector_authorized('endpoints')
if error is not None:
return error
sharedkeyauthenticated = True
payload = request.get_json(silent=True)
if not isinstance(payload, dict):
return error_response(ErrorCodes.VALIDATION_ERROR, 'No data provided')
hostname, problem = _text_field(payload, 'hostname', 255, True)
if problem is None:
serialnumber, problem = _text_field(payload, 'serialnumber', 128, True)
if problem is None:
deviceid, problem = _text_field(payload, 'deviceid', 64, False)
if problem is None:
agentversion, problem = _text_field(payload, 'agentversion', 32, False)
if problem is None:
nonce = payload.get('enrollnonce')
if not isinstance(nonce, str) or not _NONCE_PATTERN.match(nonce):
problem = 'enrollnonce must be 32-128 characters of A-Z a-z 0-9 _ -'
if problem is not None:
return error_response(ErrorCodes.VALIDATION_ERROR, problem)
now = _naive_utcnow()
sourceip = client_ip(request)
row = (EndpointEnrollRequest.query
.filter(db.func.lower(EndpointEnrollRequest.hostname) == hostname.lower(),
db.func.lower(EndpointEnrollRequest.serialnumber)
== serialnumber.lower())
.order_by(EndpointEnrollRequest.requestid.desc())
.first())
if row is None:
row = EndpointEnrollRequest(
hostname=hostname, serialnumber=serialnumber, enrollnonce=nonce,
status=EndpointEnrollRequest.STATUS_PENDING, requestedat=now)
db.session.add(row)
else:
samenonce = hmac.compare_digest(row.enrollnonce, nonce)
if row.status == EndpointEnrollRequest.STATUS_APPROVED:
row.lastseenat = now
row.sourceip = sourceip
if not samenonce:
db.session.commit()
return error_response(
CODE_ENROLL_NONCE_MISMATCH,
'This enrollment was approved for a different agent '
'installation', http_code=403)
if agentauth.active_credential_for(row.matchedassetid) is not None:
return _pending(row, REASON_ALREADY_ENROLLED, row.matchedassetid)
if row.matchedassetid is None or db.session.get(Asset, row.matchedassetid) is None:
return _pending(row, REASON_UNKNOWN_HOSTNAME)
return _issue(row, row.matchedassetid,
EndpointCredential.ENROLLED_APPROVED)
if row.status == EndpointEnrollRequest.STATUS_PENDING and not samenonce:
# First nonce wins; a second installation needs an admin.
row.lastseenat = now
row.sourceip = sourceip
return _pending(row, REASON_NONCE_CHANGED, row.matchedassetid)
if row.status == EndpointEnrollRequest.STATUS_REJECTED and samenonce:
row.lastseenat = now
row.sourceip = sourceip
db.session.commit()
return success_response(
{'status': 'pending', 'requestid': row.requestid,
'reason': REASON_REJECTED}, http_code=202)
if row.status != EndpointEnrollRequest.STATUS_PENDING:
# Issued earlier (credential since revoked) or rejected for another
# installation: a new enrollment cycle bound to this nonce.
row.enrollnonce = nonce
row.requestedat = now
row.decidedat = None
row.decidedby = None
row.lastseenat = now
row.sourceip = sourceip
if deviceid:
row.deviceid = deviceid
if agentversion:
row.agentversion = agentversion
db.session.flush()
Computer = _computer_model()
matches = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.all())
if not matches:
return _pending(row, REASON_UNKNOWN_HOSTNAME)
if len(matches) > 1:
return _pending(row, REASON_DUPLICATE_HOSTNAME)
assetid = matches[0].assetid
asset = db.session.get(Asset, assetid)
recorded = ((asset.serialnumber if asset is not None else None) or '').strip()
if not recorded:
return _pending(row, REASON_NO_SERIAL, assetid)
if recorded.lower() != serialnumber.lower():
return _pending(row, REASON_SERIAL_MISMATCH, assetid)
if agentauth.active_credential_for(assetid) is not None:
return _pending(row, REASON_ALREADY_ENROLLED, assetid)
if sharedkeyauthenticated and agentauth.auto_enroll_enabled():
return _issue(row, assetid, EndpointCredential.ENROLLED_AUTO)
return _pending(row, REASON_AWAITING_APPROVAL, assetid)
# ---------------------------------------------------------------------------
# Admin: credentials and requests
# ---------------------------------------------------------------------------
def _computers():
"""(assetid, hostname, serialnumber) for every active computer."""
Computer = _computer_model()
return (db.session.query(Computer.assetid, Computer.hostname,
Asset.serialnumber)
.join(Asset, Asset.assetid == Computer.assetid)
.filter(Asset.isactive.is_(True))
.order_by(db.func.lower(Computer.hostname))
.all())
def _pending_requests():
return (EndpointEnrollRequest.query
.filter_by(status=EndpointEnrollRequest.STATUS_PENDING)
.all())
def _pending_index(requests):
assetids = {r.matchedassetid for r in requests if r.matchedassetid}
hostnames = {(r.hostname or '').lower() for r in requests}
return assetids, hostnames
def _is_pending(assetid, hostname, index):
assetids, hostnames = index
return assetid in assetids or (hostname or '').lower() in hostnames
def _search_filter(rows, search, *keys):
if not search:
return rows
needle = search.lower()
return [row for row in rows
if any(needle in str(row.get(key) or '').lower() for key in keys)]
def _page(rows):
page, perpage = get_pagination_params()
total = len(rows)
start = (page - 1) * perpage
totalpages = (total + perpage - 1) // perpage if perpage else 0
return rows[start:start + perpage], {
'page': page, 'perpage': perpage, 'total': total,
'totalpages': totalpages, 'hasnext': page < totalpages,
'hasprev': page > 1,
}
def _settings_payload():
return {'sharedkeyallowed': agentauth.shared_key_allowed(),
'autoenroll': agentauth.auto_enroll_enabled()}
@endpoints_bp.route('/enrollment', methods=['GET'])
@require_permission('endpoints.fleet')
def get_enrollment():
"""Credential state of every computer, plus pending and recent requests."""
credentials = {c.assetid: c for c in EndpointCredential.query.all()}
pending = _pending_requests()
index = _pending_index(pending)
rows = []
for assetid, hostname, serialnumber in _computers():
credential = credentials.get(assetid)
if credential is not None and credential.revokedat is None:
state = 'enrolled'
elif _is_pending(assetid, hostname, index):
state = 'pending'
elif credential is not None:
state = 'revoked'
else:
state = 'notenrolled'
rows.append({
'assetid': assetid,
'hostname': hostname,
'serialnumber': serialnumber,
'state': state,
'credentialprefix': credential.credentialprefix if credential else None,
'issuedat': _utciso(credential.issuedat) if credential else None,
'lastusedat': _utciso(credential.lastusedat) if credential else None,
'enrolledvia': credential.enrolledvia if credential else None,
})
counts = Counter(row['state'] for row in rows)
summary = {
'enrolled': counts['enrolled'],
'notenrolled': counts['notenrolled'] + counts['revoked'],
'pending': counts['pending'],
}
state = (request.args.get('state') or '').strip().lower()
if state:
if state not in ENROLLMENT_STATES:
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Unknown state filter: {state!r}')
rows = [row for row in rows if row['state'] == state]
rows = _search_filter(rows, (request.args.get('search') or '').strip(),
'hostname', 'serialnumber')
pagerows, pagination = _page(rows)
recent = (EndpointEnrollRequest.query
.filter(EndpointEnrollRequest.status
!= EndpointEnrollRequest.STATUS_PENDING)
.order_by(EndpointEnrollRequest.lastseenat.desc())
.limit(_RECENT_REQUESTS)
.all())
requests_payload = [
r.to_dict()
for r in sorted(pending, key=lambda r: r.lastseenat or r.requestedat,
reverse=True) + recent
]
return success_response({
'settings': _settings_payload(),
'counts': summary,
'computers': pagerows,
'requests': requests_payload,
}, meta={'pagination': pagination})
@endpoints_bp.route('/enrollment/requests/<int:requestid>/approve',
methods=['POST'])
@require_role('admin')
def approve_enroll_request(requestid):
"""Bind a pending request to the asset the admin picked."""
row = db.session.get(EndpointEnrollRequest, requestid)
if row is None:
return error_response(ErrorCodes.NOT_FOUND,
f'No enroll request {requestid}', http_code=404)
if row.status == EndpointEnrollRequest.STATUS_ISSUED:
return error_response(ErrorCodes.CONFLICT,
'A credential was already issued for this request',
http_code=409)
payload = request.get_json(silent=True) or {}
assetid = payload.get('assetid')
if isinstance(assetid, bool) or not isinstance(assetid, int):
return error_response(ErrorCodes.VALIDATION_ERROR,
'assetid must be an integer')
Computer = _computer_model()
if Computer.query.filter_by(assetid=assetid).first() is None:
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Asset {assetid} is not a computer')
if agentauth.active_credential_for(assetid) is not None:
return error_response(ErrorCodes.CONFLICT,
'That computer is already enrolled; revoke its '
'credential first', http_code=409)
row.status = EndpointEnrollRequest.STATUS_APPROVED
row.matchedassetid = assetid
row.reason = None
row.decidedat = _naive_utcnow()
row.decidedby = _username()
AuditLog.log('updated', 'EndpointEnrollRequest', entityid=row.requestid,
entityname=row.hostname,
details='enroll request approved for asset {}'.format(assetid))
db.session.commit()
return success_response(row.to_dict(), message='Enroll request approved')
@endpoints_bp.route('/enrollment/requests/<int:requestid>/reject',
methods=['POST'])
@require_role('admin')
def reject_enroll_request(requestid):
row = db.session.get(EndpointEnrollRequest, requestid)
if row is None:
return error_response(ErrorCodes.NOT_FOUND,
f'No enroll request {requestid}', http_code=404)
if row.status == EndpointEnrollRequest.STATUS_ISSUED:
return error_response(ErrorCodes.CONFLICT,
'A credential was already issued for this request; '
'revoke the credential instead', http_code=409)
row.status = EndpointEnrollRequest.STATUS_REJECTED
row.decidedat = _naive_utcnow()
row.decidedby = _username()
AuditLog.log('updated', 'EndpointEnrollRequest', entityid=row.requestid,
entityname=row.hostname, details='enroll request rejected')
db.session.commit()
return success_response(row.to_dict(), message='Enroll request rejected')
@endpoints_bp.route('/enrollment/assets/<int:assetid>/revoke', methods=['POST'])
@require_role('admin')
def revoke_endpoint_credential(assetid):
"""Un-enroll a PC. Its agent gets CREDENTIAL_INVALID and enrolls again."""
credential = agentauth.active_credential_for(assetid)
if credential is None:
return error_response(ErrorCodes.NOT_FOUND,
'This computer has no active credential',
http_code=404)
credential.revokedat = _naive_utcnow()
credential.revokedby = _username()
AuditLog.log('deleted', 'EndpointCredential', entityid=assetid,
entityname=credential.credentialprefix,
details='endpoint credential {} revoked'.format(
credential.credentialprefix))
db.session.commit()
return success_response(credential.to_dict(),
message='Endpoint credential revoked')
@endpoints_bp.route('/enrollment/settings', methods=['PUT'])
@require_role('admin')
def update_enrollment_settings():
from shopdb.api import Setting
payload = request.get_json(silent=True) or {}
changes = {}
for key, settingkey in (('sharedkeyallowed', agentauth.SHARED_KEY_SETTING),
('autoenroll', agentauth.AUTO_ENROLL_SETTING)):
if key not in payload:
continue
if not isinstance(payload[key], bool):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'{key} must be true or false')
changes[key] = (settingkey, payload[key])
if not changes:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Send sharedkeyallowed and/or autoenroll')
before = _settings_payload()
for key, (settingkey, value) in changes.items():
Setting.set(settingkey, value, valuetype='boolean',
category='plugin.endpoints')
after = _settings_payload()
AuditLog.log('updated', 'Setting', entityname='endpoint enrollment',
changes={key: {'old': before[key], 'new': after[key]}
for key in changes},
details='endpoint enrollment settings changed')
db.session.commit()
return success_response(after, message='Enrollment settings saved')
# ---------------------------------------------------------------------------
# Migration of PCs already in the fleet
# ---------------------------------------------------------------------------
def _version_below(reported, target):
parsed = parse_agent_version(reported)
wanted = parse_agent_version(target)
if parsed is None or wanted is None:
return True
width = max(len(parsed), len(wanted))
return (tuple(parsed) + (0,) * (width - len(parsed))
< tuple(wanted) + (0,) * (width - len(wanted)))
def _upgrade_scriptid():
from .agentupgradescript import UPGRADE_SCRIPT_NAME
script = Script.query.filter_by(scriptname=UPGRADE_SCRIPT_NAME).first()
return script.scriptid if script is not None else None
def _store_has_package():
from . import agentstore
current = agentstore.current_version('agent')
if current is None:
return agentstore.current_path('agent') is not None
return not _version_below(current, CREDENTIAL_AGENT_VERSION)
def compute_readiness(assetids=None):
"""Migration state for every active computer (or just assetids).
Returns a list of dicts ordered by hostname: assetid, hostname,
agentversion, lastreportedat, state, blockers.
"""
computers = _computers()
hostnamecounts = Counter((hostname or '').lower()
for _assetid, hostname, _serial in computers)
if assetids is not None:
wanted = set(assetids)
computers = [row for row in computers if row[0] in wanted]
if not computers:
return []
ids = [row[0] for row in computers]
active = {c.assetid for c in EndpointCredential.query
.filter(EndpointCredential.revokedat.is_(None)).all()}
index = _pending_index(_pending_requests())
states = {s.assetid: s for s in
EndpointState.query.filter(EndpointState.assetid.in_(ids)).all()}
scriptid = _upgrade_scriptid()
upgrading = set()
if scriptid is not None:
upgrading = {row[0] for row in db.session.query(EndpointCommand.assetid)
.filter(EndpointCommand.assetid.in_(ids),
EndpointCommand.action == 'run_script',
EndpointCommand.target == f'script:{scriptid}',
EndpointCommand.status.in_(
(EndpointCommand.STATUS_PENDING,
EndpointCommand.STATUS_RUNNING)))}
haspackage = _store_has_package()
minimum = minimum_agent_version_for('run_script')
offlinecutoff = _naive_utcnow() - _OFFLINE_AFTER
result = []
for assetid, hostname, serialnumber in computers:
state = states.get(assetid)
agentversion = state.agentversion if state is not None else None
lastreportedat = state.lastreportedat if state is not None else None
blockers = []
if assetid in active:
status = 'enrolled'
elif _is_pending(assetid, hostname, index):
status = 'pending'
elif assetid in upgrading:
status = 'upgrading'
elif state is None:
status = 'notreporting'
else:
if lastreportedat is None or lastreportedat < offlinecutoff:
blockers.append(BLOCKER_OFFLINE)
if not (serialnumber or '').strip():
blockers.append(BLOCKER_NO_SERIAL)
if hostnamecounts[(hostname or '').lower()] > 1:
blockers.append(BLOCKER_DUPLICATE_HOSTNAME)
if ACTIVE_LOCK_FOR(assetid) is not None:
blockers.append(BLOCKER_LOCKED)
if not haspackage:
blockers.append(BLOCKER_NO_PACKAGE)
if minimum and not agent_meets_minimum(agentversion, minimum):
blockers.append(BLOCKER_TOO_OLD)
if not _version_below(agentversion, CREDENTIAL_AGENT_VERSION):
blockers.append(BLOCKER_AWAITING_ENROLLMENT)
if not blockers and 'run_script' not in ALLOWED_ACTIONS_FOR(
assetid=assetid):
blockers.append(BLOCKER_POLICY)
status = 'blocked' if blockers else 'ready'
result.append({
'assetid': assetid,
'hostname': hostname,
'agentversion': agentversion,
'lastreportedat': _utciso(lastreportedat),
'state': status,
'blockers': blockers,
})
return result
def _wave_progress(wave, scriptid):
members = wave.assetidlist
enrolled = set()
failed = set()
queued = set()
if members:
enrolled = {c.assetid for c in EndpointCredential.query.filter(
EndpointCredential.assetid.in_(members),
EndpointCredential.revokedat.is_(None)).all()}
if scriptid is not None:
commands = (EndpointCommand.query
.filter(EndpointCommand.assetid.in_(members),
EndpointCommand.action == 'run_script',
EndpointCommand.target == f'script:{scriptid}',
EndpointCommand.createdat >= wave.createdat)
.order_by(EndpointCommand.createdat.asc())
.all())
latest = {}
for command in commands:
queued.add(command.assetid)
latest[command.assetid] = command
failed = {assetid for assetid, command in latest.items()
if assetid not in enrolled
and command.status in (EndpointCommand.STATUS_FAILED,
EndpointCommand.STATUS_EXPIRED)}
return {
'waveid': wave.waveid,
'createdat': _utciso(wave.createdat),
'createdby': wave.createdby,
'size': len(members),
'queued': len(queued),
'enrolled': len(enrolled),
'failed': len(failed),
'done': len(enrolled) + len(failed) >= len(members),
}
@endpoints_bp.route('/enrollment/migration', methods=['GET'])
@require_permission('endpoints.fleet')
def get_enrollment_migration():
"""Per-computer readiness to move onto per-PC credentials, plus waves."""
rows = compute_readiness()
counts = Counter(row['state'] for row in rows)
state = (request.args.get('state') or '').strip().lower()
if state:
if state not in MIGRATION_STATES:
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Unknown state filter: {state!r}')
rows = [row for row in rows if row['state'] == state]
rows = _search_filter(rows, (request.args.get('search') or '').strip(),
'hostname', 'agentversion')
pagerows, pagination = _page(rows)
scriptid = _upgrade_scriptid()
waves = (EndpointMigrationWave.query
.order_by(EndpointMigrationWave.createdat.desc())
.limit(_RECENT_REQUESTS)
.all())
return success_response({
'target': CREDENTIAL_AGENT_VERSION,
'counts': {name: counts[name] for name in MIGRATION_STATES},
'computers': pagerows,
'waves': [_wave_progress(wave, scriptid) for wave in waves],
}, meta={'pagination': pagination})
def _response_message(response):
try:
body = response.get_json() or {}
except Exception: # noqa: BLE001 - reason text only
return 'enqueue refused'
error = ((body.get('data') or {}).get('error') or {})
return error.get('message') or body.get('message') or 'enqueue refused'
@endpoints_bp.route('/enrollment/migration/wave', methods=['POST'])
@require_role('admin')
def create_migration_wave():
"""Push the agent upgrade to a batch of READY PCs.
Body {"assetids": [...]} or {"next": n}. Readiness is re-checked per PC; the
agent is packaged once for the whole wave, then each PC is queued through
the same path as POST /<asset>/agentupgrade.
"""
from .routes import prepare_agent_upgrade, queue_prepared_agent_upgrade
payload = request.get_json(silent=True) or {}
requested = payload.get('assetids')
nextcount = payload.get('next')
if requested is not None:
if (not isinstance(requested, list) or not requested
or not all(isinstance(v, int) and not isinstance(v, bool)
for v in requested)):
return error_response(ErrorCodes.VALIDATION_ERROR,
'assetids must be a non-empty list of integers')
if len(requested) > _WAVE_MAXIMUM:
return error_response(ErrorCodes.VALIDATION_ERROR,
f'A wave holds at most {_WAVE_MAXIMUM} PCs')
candidates = list(dict.fromkeys(requested))
elif nextcount is not None:
if (isinstance(nextcount, bool) or not isinstance(nextcount, int)
or nextcount < 1 or nextcount > _WAVE_MAXIMUM):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'next must be an integer from 1 to {_WAVE_MAXIMUM}')
candidates = [row['assetid'] for row in compute_readiness()
if row['state'] == 'ready'][:nextcount]
else:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Send assetids or next')
readiness = {row['assetid']: row for row in compute_readiness(candidates)}
skipped = []
ready = []
for assetid in candidates:
row = readiness.get(assetid)
if row is None:
skipped.append({'assetid': assetid, 'reason': 'not a computer'})
elif row['state'] != 'ready':
reason = ', '.join(row['blockers']) if row['blockers'] else row['state']
skipped.append({'assetid': assetid, 'reason': reason})
else:
ready.append(assetid)
queued = []
waveid = None
if ready:
startedat = _naive_utcnow().replace(microsecond=0)
registration, seeded, error = prepare_agent_upgrade()
if error is not None:
return error
for assetid in ready:
response = queue_prepared_agent_upgrade(assetid, registration,
seeded, {})
if getattr(response, 'status_code', None) == 201:
queued.append(assetid)
else:
db.session.rollback()
skipped.append({'assetid': assetid,
'reason': _response_message(response)})
if queued:
wave = EndpointMigrationWave(
createdat=startedat, createdby=_username(),
assetids=','.join(str(v) for v in queued))
db.session.add(wave)
db.session.flush()
waveid = wave.waveid
AuditLog.log('created', 'EndpointMigrationWave', entityid=waveid,
entityname='agent {}'.format(registration['agentversion']),
details='migration wave queued agent upgrade for {} PC(s): {}'
.format(len(queued), wave.assetids))
db.session.commit()
return success_response(
{'waveid': waveid, 'queued': queued, 'skipped': skipped},
message='{} PC(s) queued'.format(len(queued)),
http_code=201 if queued else 200)

File diff suppressed because it is too large Load Diff

View File

@@ -1,10 +1,10 @@
"""Add fingerprint column for delta-based application updates
Revision ID: 0020
Revises: 0019
Revision ID: endpoints0020appfingerprints
Revises: endpoints0019splitpolicies
Create Date: 2026-09-14
Performance optimization: Add app_fingerprint column to endpointapplications
Performance optimization: Add appfingerprint column to endpointapplications
table to enable delta-based updates instead of delete-all-insert-all pattern.
Impact:
@@ -35,6 +35,10 @@ SAFETY FEATURES:
- Delete count verification detects silent failures
- Memory limits prevent pathological cases (>5000 apps)
- Metrics logged for monitoring
Idempotent: a database that already has the column (create_all built it from
the model, or an earlier build of this revision added it as app_fingerprint)
is left alone. 0023 renames the old spelling.
"""
from alembic import op
@@ -47,26 +51,40 @@ branch_labels = None
depends_on = None
plugin_name = 'endpoints'
TABLE = 'endpointapplications'
INDEX = 'idx_endpointapplications_fingerprint'
def _columns():
from sqlalchemy import inspect
return {c['name'] for c in inspect(op.get_bind()).get_columns(TABLE)}
def _indexes():
from sqlalchemy import inspect
return {ix['name'] for ix in inspect(op.get_bind()).get_indexes(TABLE)}
def upgrade():
# Add app_fingerprint column - nullable for backward compatibility during rollout
op.add_column('endpointapplications',
sa.Column('app_fingerprint', sa.String(64), nullable=True))
# Add UNIQUE index for fast fingerprint lookups (assetid, app_fingerprint)
# This index enables efficient delta detection: "which fingerprints exist for this asset?"
# UNIQUE constraint ensures one fingerprint per asset (delta logic requirement)
op.create_index(
'idx_endpointapplications_fingerprint',
'endpointapplications',
['assetid', 'app_fingerprint'],
unique=True
)
columns = _columns()
# Old spelling present: leave it for 0023 to rename.
if 'appfingerprint' in columns or 'app_fingerprint' in columns:
return
op.add_column(TABLE, sa.Column('appfingerprint', sa.String(64),
nullable=True))
if INDEX not in _indexes():
# Unique per asset: the delta logic keys rows on it.
op.create_index(INDEX, TABLE, ['assetid', 'appfingerprint'],
unique=True)
def downgrade():
# Remove index first
op.drop_index('idx_endpointapplications_fingerprint', 'endpointapplications')
# Remove column
op.drop_column('endpointapplications', 'app_fingerprint')
if INDEX in _indexes():
op.drop_index(INDEX, table_name=TABLE)
columns = _columns()
for name in ('appfingerprint', 'app_fingerprint'):
if name in columns:
with op.batch_alter_table(TABLE) as batch:
batch.drop_column(name)

View File

@@ -1,42 +1,26 @@
"""Add client certificate authentication columns
"""Retired: client certificate columns on assets.
Revision ID: endpoints0021clientcerts
Revises: endpoints0020appfingerprints
Create Date: 2026-09-14
This revision once added client certificate columns to the core assets table.
Client certificate authentication was replaced by per-PC endpoint credentials
(0023), so the revision is kept only so databases already stamped with it
still resolve. It does nothing; 0023 drops the columns wherever an earlier
build of this revision created them.
"""
from alembic import op
import sqlalchemy as sa
revision = 'endpoints0021clientcerts'
down_revision = 'endpoints0020appfingerprints' # Will resolve to actual revision ID
down_revision = 'endpoints0020appfingerprints'
branch_labels = None
depends_on = None
plugin_name = 'endpoints'
def upgrade():
# Add client cert columns to assets table
op.add_column('assets',
sa.Column('client_cert_fingerprint', sa.String(64), nullable=True))
op.add_column('assets',
sa.Column('cert_issued_at', sa.DateTime, nullable=True))
op.add_column('assets',
sa.Column('cert_expires_at', sa.DateTime, nullable=True))
# Add index for fast fingerprint lookup during authentication
op.create_index(
'idx_assets_cert_fingerprint',
'assets',
['client_cert_fingerprint'],
unique=False
)
pass
def downgrade():
op.drop_index('idx_assets_cert_fingerprint', 'assets')
op.drop_column('assets', 'cert_expires_at')
op.drop_column('assets', 'cert_issued_at')
op.drop_column('assets', 'client_cert_fingerprint')
pass

View File

@@ -1,12 +1,13 @@
"""Add certificate revocation columns to assets
"""Retired: client certificate revocation columns on assets.
Revision ID: endpoints0022certrevocation
Revises: endpoints0021clientcerts
Create Date: 2026-09-14
"""
from alembic import op
import sqlalchemy as sa
Kept only so databases already stamped with this revision still resolve. It
does nothing; 0023 drops the columns wherever an earlier build of this
revision created them.
"""
revision = 'endpoints0022certrevocation'
down_revision = 'endpoints0021clientcerts'
@@ -16,16 +17,8 @@ plugin_name = 'endpoints'
def upgrade():
# Add revocation tracking columns to assets table
op.add_column('assets',
sa.Column('cert_revoked_at', sa.DateTime, nullable=True,
comment='When the certificate was revoked (null = not revoked)'))
op.add_column('assets',
sa.Column('cert_revocation_reason', sa.String(255), nullable=True,
comment='Why the certificate was revoked (admin note)'))
pass
def downgrade():
op.drop_column('assets', 'cert_revocation_reason')
op.drop_column('assets', 'cert_revoked_at')
pass

View File

@@ -0,0 +1,167 @@
"""Per-PC endpoint credentials; remove the client certificate leftovers.
Revision ID: endpoints0023credentials
Revises: endpoints0022certrevocation
Create Date: 2026-09-15
Three things, each guarded against the live schema so the revision is safe on
a database built by create_all, on one that ran the earlier client certificate
builds of 0021/0022, and on a re-run:
1. Drop the client certificate columns (and their index) from the core assets
table where an earlier build of 0021/0022 created them.
2. Rename endpointapplications.app_fingerprint to appfingerprint where an
earlier build of 0020 used the underscore spelling.
3. Create endpointcredentials, endpointenrollrequests and
endpointmigrationwaves.
MySQL 5.6 safe: no JSON columns, and the enroll request lookup index uses
prefix lengths so the key stays under 767 bytes on utf8mb4.
Downgrade drops the three tables. The certificate columns are not restored:
the code that read them is gone.
"""
from alembic import op
import sqlalchemy as sa
revision = 'endpoints0023credentials'
down_revision = 'endpoints0022certrevocation'
branch_labels = None
depends_on = None
plugin_name = 'endpoints'
CREDENTIALS = 'endpointcredentials'
REQUESTS = 'endpointenrollrequests'
WAVES = 'endpointmigrationwaves'
_CERT_COLUMNS = ('client_cert_fingerprint', 'cert_issued_at', 'cert_expires_at',
'cert_revoked_at', 'cert_revocation_reason')
_APPS = 'endpointapplications'
_FINGERPRINT_INDEX = 'idx_endpointapplications_fingerprint'
def _inspector():
from sqlalchemy import inspect
return inspect(op.get_bind())
def _has_table(name):
return _inspector().has_table(name)
def _columns(table):
return {c['name'] for c in _inspector().get_columns(table)}
def _indexes(table):
return _inspector().get_indexes(table)
def _drop_certificate_columns():
if not _has_table('assets'):
return
present = _columns('assets') & set(_CERT_COLUMNS)
if not present:
return
# Any index over a certificate column goes first: 0021 named its own,
# create_all names by convention, and SQLite refuses to drop an indexed column.
for index in _indexes('assets'):
if set(index.get('column_names') or ()) & present:
op.drop_index(index['name'], table_name='assets')
for column in _CERT_COLUMNS:
if column in present:
op.drop_column('assets', column)
def _rename_fingerprint():
if not _has_table(_APPS):
return
columns = _columns(_APPS)
if 'app_fingerprint' not in columns:
return
if 'appfingerprint' in columns:
return
for index in _indexes(_APPS):
if 'app_fingerprint' in (index.get('column_names') or ()):
op.drop_index(index['name'], table_name=_APPS)
op.alter_column(_APPS, 'app_fingerprint', new_column_name='appfingerprint',
existing_type=sa.String(64), existing_nullable=True)
op.create_index(_FINGERPRINT_INDEX, _APPS, ['assetid', 'appfingerprint'],
unique=True)
def upgrade():
_drop_certificate_columns()
_rename_fingerprint()
if not _has_table(CREDENTIALS):
op.create_table(
CREDENTIALS,
sa.Column('credentialid', sa.Integer(), nullable=False),
sa.Column('assetid', sa.Integer(), nullable=False),
sa.Column('credentialhash', sa.CHAR(length=64), nullable=False),
sa.Column('credentialprefix', sa.String(length=12), nullable=False),
sa.Column('enrolledvia', sa.String(length=16), nullable=False),
sa.Column('issuedat', sa.DateTime(), nullable=False),
sa.Column('lastusedat', sa.DateTime(), nullable=True),
sa.Column('revokedat', sa.DateTime(), nullable=True),
sa.Column('revokedby', sa.String(length=64), nullable=True),
sa.ForeignKeyConstraint(['assetid'], ['assets.assetid'],
ondelete='CASCADE'),
sa.PrimaryKeyConstraint('credentialid'),
sa.UniqueConstraint('assetid'),
sa.UniqueConstraint('credentialhash'),
)
if not _has_table(REQUESTS):
op.create_table(
REQUESTS,
sa.Column('requestid', sa.Integer(), nullable=False),
sa.Column('hostname', sa.String(length=255), nullable=False),
sa.Column('serialnumber', sa.String(length=128), nullable=False),
sa.Column('deviceid', sa.String(length=64), nullable=True),
sa.Column('enrollnonce', sa.String(length=128), nullable=False),
sa.Column('agentversion', sa.String(length=32), nullable=True),
sa.Column('status', sa.String(length=16), nullable=False),
sa.Column('reason', sa.String(length=64), nullable=True),
sa.Column('matchedassetid', sa.Integer(), nullable=True),
sa.Column('sourceip', sa.String(length=45), nullable=True),
sa.Column('requestedat', sa.DateTime(), nullable=False),
sa.Column('lastseenat', sa.DateTime(), nullable=False),
sa.Column('decidedat', sa.DateTime(), nullable=True),
sa.Column('decidedby', sa.String(length=64), nullable=True),
sa.ForeignKeyConstraint(['matchedassetid'], ['assets.assetid'],
ondelete='SET NULL'),
sa.PrimaryKeyConstraint('requestid'),
)
existing = {tuple(ix.get('column_names') or ()) for ix in _indexes(REQUESTS)}
if ('hostname', 'serialnumber') not in existing:
op.create_index('idx_endpointenrollrequests_hostserial', REQUESTS,
['hostname', 'serialnumber'],
mysql_length={'hostname': 128, 'serialnumber': 63})
if not _has_table(WAVES):
op.create_table(
WAVES,
sa.Column('waveid', sa.Integer(), nullable=False),
sa.Column('createdat', sa.DateTime(), nullable=False),
sa.Column('createdby', sa.String(length=64), nullable=True),
sa.Column('assetids', sa.Text(), nullable=False),
sa.Column('notes', sa.String(length=255), nullable=True),
sa.PrimaryKeyConstraint('waveid'),
)
def downgrade():
if _has_table(WAVES):
op.drop_table(WAVES)
if _has_table(REQUESTS):
for index in _indexes(REQUESTS):
if index['name'] == 'idx_endpointenrollrequests_hostserial':
op.drop_index(index['name'], table_name=REQUESTS)
op.drop_table(REQUESTS)
if _has_table(CREDENTIALS):
op.drop_table(CREDENTIALS)

View File

@@ -6,6 +6,9 @@ from .endpoint import (
)
from .monitor import EndpointMonitor
from .operatorreport import OperatorReport
from .credential import (
EndpointCredential, EndpointEnrollRequest, EndpointMigrationWave,
)
from .health import (
EndpointHealthState, EndpointHealthTransition, EndpointMonitorImpact,
)
@@ -14,5 +17,6 @@ __all__ = [
'EndpointState', 'EndpointEvent', 'EndpointCommand', 'EndpointApplication',
'EndpointLock', 'EndpointLogCollection',
'EndpointMonitor', 'OperatorReport',
'EndpointCredential', 'EndpointEnrollRequest', 'EndpointMigrationWave',
'EndpointHealthState', 'EndpointHealthTransition', 'EndpointMonitorImpact',
]

View File

@@ -0,0 +1,148 @@
"""Per-PC endpoint credentials, enrollment requests and migration waves.
EndpointCredential
One row per asset. Holds only the sha256 of the secret the
agent was issued, plus a short display prefix. Revoking sets
revokedat; re-issuing after a revoke UPDATES the same row, so
an asset never carries two credentials.
EndpointEnrollRequest
One row per (hostname, serial number) that asked to enroll. The
agent's enrollment nonce is recorded on first sight and is what
an admin approval is bound to, so a second machine claiming the
same hostname and serial cannot collect the approved credential.
EndpointMigrationWave
An admin action that pushed the agent upgrade to a set of PCs.
Progress is computed live from credentials and commands; only
the membership is stored.
"""
from datetime import datetime
from shopdb.api import db
def _utciso(value):
return value.isoformat() + 'Z' if value else None
class EndpointCredential(db.Model):
"""The credential issued to one endpoint (hash only)."""
__tablename__ = 'endpointcredentials'
ENROLLED_AUTO = 'auto'
ENROLLED_APPROVED = 'approved'
credentialid = db.Column(db.Integer, primary_key=True)
assetid = db.Column(
db.Integer,
db.ForeignKey('assets.assetid', ondelete='CASCADE'),
nullable=False,
unique=True,
)
credentialhash = db.Column(db.CHAR(64), nullable=False, unique=True)
credentialprefix = db.Column(db.String(12), nullable=False)
enrolledvia = db.Column(db.String(16), nullable=False)
issuedat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
lastusedat = db.Column(db.DateTime, nullable=True)
revokedat = db.Column(db.DateTime, nullable=True)
revokedby = db.Column(db.String(64), nullable=True)
@property
def isactive(self):
return self.revokedat is None
def to_dict(self):
return {
'credentialid': self.credentialid,
'assetid': self.assetid,
'credentialprefix': self.credentialprefix,
'enrolledvia': self.enrolledvia,
'issuedat': _utciso(self.issuedat),
'lastusedat': _utciso(self.lastusedat),
'revokedat': _utciso(self.revokedat),
'revokedby': self.revokedby,
}
def __repr__(self):
return f'<EndpointCredential asset={self.assetid} {self.credentialprefix}>'
class EndpointEnrollRequest(db.Model):
"""A PC asking for a credential."""
__tablename__ = 'endpointenrollrequests'
STATUS_PENDING = 'pending'
STATUS_APPROVED = 'approved'
STATUS_REJECTED = 'rejected'
STATUS_ISSUED = 'issued'
requestid = db.Column(db.Integer, primary_key=True)
hostname = db.Column(db.String(255), nullable=False)
serialnumber = db.Column(db.String(128), nullable=False)
deviceid = db.Column(db.String(64), nullable=True)
enrollnonce = db.Column(db.String(128), nullable=False)
agentversion = db.Column(db.String(32), nullable=True)
status = db.Column(db.String(16), nullable=False, default=STATUS_PENDING)
reason = db.Column(db.String(64), nullable=True)
matchedassetid = db.Column(
db.Integer,
db.ForeignKey('assets.assetid', ondelete='SET NULL'),
nullable=True,
)
sourceip = db.Column(db.String(45), nullable=True)
requestedat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
lastseenat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
decidedat = db.Column(db.DateTime, nullable=True)
decidedby = db.Column(db.String(64), nullable=True)
# Prefix lengths keep the key under 767 bytes on utf8mb4 (4 bytes a char).
__table_args__ = (
db.Index('idx_endpointenrollrequests_hostserial', 'hostname',
'serialnumber',
mysql_length={'hostname': 128, 'serialnumber': 63}),
)
def to_dict(self):
return {
'requestid': self.requestid,
'hostname': self.hostname,
'serialnumber': self.serialnumber,
'deviceid': self.deviceid,
'agentversion': self.agentversion,
'reason': self.reason,
'status': self.status,
'matchedassetid': self.matchedassetid,
'requestedat': _utciso(self.requestedat),
'lastseenat': _utciso(self.lastseenat),
'decidedat': _utciso(self.decidedat),
'decidedby': self.decidedby,
'sourceip': self.sourceip,
}
def __repr__(self):
return f'<EndpointEnrollRequest {self.hostname!r} {self.status}>'
class EndpointMigrationWave(db.Model):
"""One batch of agent upgrades pushed to move PCs onto credentials."""
__tablename__ = 'endpointmigrationwaves'
waveid = db.Column(db.Integer, primary_key=True)
createdat = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
createdby = db.Column(db.String(64), nullable=True)
# Comma list of asset ids. TEXT, not JSON: MySQL 5.6.
assetids = db.Column(db.Text, nullable=False, default='')
notes = db.Column(db.String(255), nullable=True)
@property
def assetidlist(self):
return [int(part) for part in (self.assetids or '').split(',')
if part.strip().isdigit()]
def __repr__(self):
return f'<EndpointMigrationWave {self.waveid}>'

View File

@@ -468,11 +468,11 @@ class EndpointApplication(db.Model):
# publisher. Allows efficient comparison to detect changes without comparing all
# columns. Computed by agent and sent in payload. Nullable for backward compatibility
# during migration - backfill script will populate existing rows.
app_fingerprint = db.Column(db.String(64), nullable=True)
appfingerprint = db.Column(db.String(64), nullable=True)
__table_args__ = (
db.Index('idx_endpointapplications_assetname', 'assetid', 'displayname'),
db.Index('idx_endpointapplications_fingerprint', 'assetid', 'app_fingerprint', unique=True),
db.Index('idx_endpointapplications_fingerprint', 'assetid', 'appfingerprint', unique=True),
)
def __repr__(self):

View File

@@ -20,7 +20,7 @@ from datetime import datetime, timezone
from pathlib import Path
from typing import Dict, List, Optional, Type
from flask import Flask, Blueprint
from flask import Flask, Blueprint, g
from shopdb.plugins.base import BasePlugin, PluginMeta
from shopdb.api import db
@@ -298,6 +298,25 @@ class EndpointsPlugin(BasePlugin):
'key is provisioned so revocation is a real kill '
'switch.',
},
{
'key': 'plugin.endpoints.sharedkeyallowed',
'value': 'true',
'valuetype': 'boolean',
'category': 'plugin.endpoints',
'description': 'Accept the shared fleet collector key from PCs '
'that have not enrolled. Enrolled PCs are always '
'refused it. Turn off once every PC has a '
'per-PC credential.',
},
{
'key': 'plugin.endpoints.autoenroll',
'value': 'true',
'valuetype': 'boolean',
'category': 'plugin.endpoints',
'description': 'Issue a per-PC credential without admin '
'approval when the hostname and serial number '
'match exactly one computer.',
},
{
'key': 'plugin.endpoints.agentkeyprovisioned',
'value': 'false',
@@ -351,6 +370,15 @@ class EndpointsPlugin(BasePlugin):
'Windows trusted roots.',
'position': 29,
},
{
'group': 'Endpoints',
'to': '/settings/endpoints/enrollment',
'icon': 'key',
'title': 'Enrollment',
'description': 'Move agents to per-PC credentials, approve '
'enrollment requests, and retire the shared key.',
'position': 30,
},
]
def get_navigation_items(self) -> List[Dict]:
@@ -804,9 +832,18 @@ class EndpointsPlugin(BasePlugin):
# ---- ADR-006 collector contract -------------------------------------
def resolve_agent_credential(self, secret: str) -> Optional[int]:
from .api import agentauth
return agentauth.resolve_credential(secret)
def agent_shared_key_refused(self, hostname: str) -> bool:
from .api import agentauth
return agentauth.shared_key_refused_for(hostname)
def get_collector_schema(self) -> Optional[Dict]:
return {
'identityfield': 'hostname',
'acceptsagentcredential': True,
'fields': {
'type': 'object',
'required': ['hostname'],
@@ -1086,7 +1123,7 @@ class EndpointsPlugin(BasePlugin):
displayname=displayname[:255],
displayversion=_clamp(item.get('displayversion'), 128),
publisher=_clamp(item.get('publisher'), 255),
app_fingerprint=None, # Agent didn't send it
appfingerprint=None, # Agent didn't send it
))
stored += 1
@@ -1153,7 +1190,7 @@ class EndpointsPlugin(BasePlugin):
displayname=app_data['displayname'],
displayversion=app_data['displayversion'],
publisher=app_data['publisher'],
app_fingerprint=fp,
appfingerprint=fp,
))
stored += 1
@@ -1164,14 +1201,14 @@ class EndpointsPlugin(BasePlugin):
)
return stored
current_fps = {app.app_fingerprint for app in current_apps
if app.app_fingerprint} # Skip NULL fingerprints from migration
current_fps = {app.appfingerprint for app in current_apps
if app.appfingerprint} # Skip NULL fingerprints from migration
# TRANSITION: If we have apps without fingerprints, this is the first report
# after agent upgrade. Delete all NULL-fingerprint apps to prevent duplicates.
# SAFETY: Validate incoming apps BEFORE deleting legacy apps to prevent
# leaving asset with zero apps if transaction aborts.
null_fp_apps = [app for app in current_apps if app.app_fingerprint is None]
null_fp_apps = [app for app in current_apps if app.appfingerprint is None]
if null_fp_apps:
if not incoming_fps:
warnings.append(
@@ -1192,7 +1229,7 @@ class EndpointsPlugin(BasePlugin):
if to_remove:
deleted_count = EndpointApplication.query.filter(
EndpointApplication.assetid == assetid,
EndpointApplication.app_fingerprint.in_(list(to_remove))
EndpointApplication.appfingerprint.in_(list(to_remove))
).delete(synchronize_session=False)
# SAFETY: Verify delete count matches expected. Log as WARNING
@@ -1215,7 +1252,7 @@ class EndpointsPlugin(BasePlugin):
displayname=app_data['displayname'],
displayversion=app_data['displayversion'],
publisher=app_data['publisher'],
app_fingerprint=fp,
appfingerprint=fp,
))
# SAFETY: Flush to detect constraint violations before continuing
@@ -1240,6 +1277,14 @@ class EndpointsPlugin(BasePlugin):
if not hostname:
raise ValueError('hostname is required')
from .api import agentauth
if agentauth.is_credential_caller():
# A credentialed agent reports only for its own asset, and never
# adopts: its asset already exists.
if agentauth.resolve_hostname(hostname) != g.agentassetid:
raise PermissionError(
'The payload names a different endpoint than the credential')
computer = (Computer.query
.filter(db.func.lower(Computer.hostname) == hostname.lower())
.first())
@@ -1270,21 +1315,6 @@ class EndpointsPlugin(BasePlugin):
assetid = computer.assetid
# Authorization: if authenticated via client certificate, verify the
# certificate's asset matches the payload's hostname asset. Without this
# check an agent with a valid cert for asset A can submit telemetry for
# asset B by changing the hostname field.
from flask import g
if hasattr(g, 'cert_authenticated_asset_id'):
if g.cert_authenticated_asset_id != assetid:
self.logger.warning(
'[collector] AUTHORIZATION DENIED: Certificate authenticates '
'asset %d but payload hostname %s resolves to asset %d',
g.cert_authenticated_asset_id, hostname, assetid)
raise PermissionError(
f'Certificate authenticates asset {g.cert_authenticated_asset_id} '
f'but payload reports for asset {assetid}')
state = EndpointState.query.filter_by(assetid=assetid).first()
action = 'updated'
if state is None:
@@ -1510,12 +1540,18 @@ class EndpointsPlugin(BasePlugin):
# Fail-open, matching the reconciler above: a health error must never
# fail an otherwise-good check-in. The previous verdict simply stands,
# which is correct - a failed evaluation is not evidence of health.
#
# SAVEPOINT: a failed flush (IntegrityError and friends) otherwise
# leaves the session in a rolled-back-needed state, and the caught
# exception would still sink the outer commit of the whole check-in.
# Rolling back only the savepoint keeps the telemetry flushed above.
try:
from .services.health import evaluate_endpoint_health
evaluate_endpoint_health(
assetid, state,
computertypeid=getattr(computer, 'computertypeid', None))
db.session.flush()
with db.session.begin_nested():
evaluate_endpoint_health(
assetid, state,
computertypeid=getattr(computer, 'computertypeid', None))
db.session.flush()
except Exception:
logger.exception(
'health evaluation failed for asset %s; leaving verdict '