ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add endpoint locks and split endpoints.deploy into three grants

Browse Source 
Endpoint locks
--------------
Adds EndpointLock so an admin can hold an endpoint against automated and
operator action, with migration 0018 and enforcement in the enqueue path
rather than only in the UI, so a lock cannot be bypassed by calling the
API directly.

Split endpoints.deploy
----------------------
endpoints.deploy bundled three actions of very different risk behind a
single name:

	scheduledtask_run -> starts a task already registered and allowlisted
	update_app        -> runs a pinned installer
	run_script        -> arbitrary code as SYSTEM unless the descriptor
						 pins a user context

Because they shared one grant there was no way to let an operator push an
application update without also handing them arbitrary SYSTEM code
execution on every endpoint they could reach. The permission name
described the lightest of the three while silently carrying the heaviest.

commands.ELEVATED_ACTION_PERMISSIONS now maps each action to its own
permission (endpoints.corptools, endpoints.updateapp,
endpoints.runscript). The enqueue route reads that table, so no route
logic changed. Migration 7d42 preserves existing access exactly: anyone
holding endpoints.deploy receives all three replacements.
This commit is contained in:
benjamin.baker
2026-09-07 12:36:45 -04:00
parent 879451b689
commit 627d3f5ff5
8 changed files with 1067 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

358
plugins/endpoints/api/routes.py
View File

@@ -19,9 +19,10 @@ from flask_jwt_extended import current_user
from shopdb.api import (
db, Asset, Setting, success_response, error_response, paginated_response,
ErrorCodes, get_pagination_params, paginate_query, require_permission,
require_role, caller_holds, mint_service_token, list_service_tokens,
require_role, caller_holds, caller_is_role, mint_service_token,
list_service_tokens,
revoke_service_tokens,
AuditLog, collector_authorized, User,
AuditLog, audit_log, collector_authorized, User,
Application, AppVersion, Script,
)
@@ -29,7 +30,7 @@ from ..models import (
EndpointState, EndpointEvent, EndpointCommand, EndpointApplication,
EndpointHealthState, OperatorReport,
)
from ..commands import COMMAND_ALLOWLIST, is_allowed_action, ALLOWED_ACTIONS_FOR, row_actions_for, requires_target, action_targetkind, elevated_permission_for
from ..commands import COMMAND_ALLOWLIST, is_allowed_action, ALLOWED_ACTIONS_FOR, row_actions_for, requires_target, action_targetkind, elevated_permission_for, PLACE_POLICY_HOLD, ACTIVE_POLICY_HOLDS, HOLDABLE_ACTIONS_FOR, RELEASE_POLICY_HOLD, ACTIVE_LOCK_FOR, PLACE_ENDPOINT_LOCK, RELEASE_ENDPOINT_LOCK
from ..eventcatalog import CRITICAL_EVENT_IDS, event_severity, event_title
endpoints_bp = Blueprint('endpoints', __name__)
@@ -2108,14 +2109,286 @@ def list_endpoint_commands(asset_id: int):
return success_response([row.to_dict() for row in rows])
@endpoints_bp.route('/<int:asset_id>/policyholds', methods=['GET'])
@require_permission('endpoints.view')
def list_endpoint_policy_holds(asset_id: int):
"""Active break-glass holds for one endpoint.
Gated by endpoints.view, not by the admin role that places a hold. A hold
suppresses reconciliation, so an operator watching a machine drift from its
policy needs to see that the drift is deliberate, who authorized it and when
it lapses. Hiding that from non-admins turns an audited override back into
unexplained behavior.
"""
return success_response(ACTIVE_POLICY_HOLDS(asset_id))
@endpoints_bp.route('/<int:asset_id>/holdableactions', methods=['GET'])
@require_permission('endpoints.view')
def list_endpoint_holdable_actions(asset_id: int):
"""Desired-mode actions on this endpoint, with current hold state.
Feeds the admin hold picker. Read-gated rather than admin-gated to match the
holds list: knowing WHICH actions this endpoint reconciles is the same class
of provenance as knowing which are currently held.
"""
return success_response(HOLDABLE_ACTIONS_FOR(asset_id))
@endpoints_bp.route('/<int:asset_id>/policyholds', methods=['POST'])
@require_permission('endpoints.command')
def place_endpoint_policy_hold(asset_id: int):
"""Deliberately suspend desired-state reconciliation for one or more actions.
The same authority as break-glass, reached directly instead of as a side
effect of a denied command: an admin who knows they are about to work on a
machine should not have to trip a policy denial first to stop the reconciler
fighting them.
Accepts `actions` (a list) or `action` (a single name, kept so existing
callers keep working). Every named action is validated BEFORE anything is
written, and one bad name rejects the whole request. Partial success on a
break-glass control is the wrong failure mode - "three of your five holds
landed" is worthless to someone about to take a machine apart, and the
missing two are discovered only when the reconciler undoes their work.
One shared window covers every action in the request: holding five actions
means doing one job on one machine, and that job has one duration.
Admin-only via caller_is_role for the same reason as the override branch - a
scoped personal token is a narrower caller than its owner, so a leaked agent
token must not be able to switch off policy enforcement.
"""
if not caller_is_role('admin'):
return error_response(ErrorCodes.FORBIDDEN,
'Placing a policy hold requires the admin role',
http_code=403)
payload = request.get_json(silent=True) or {}
reason = (payload.get('reason') or '').strip()
raw = payload.get('actions')
if raw is None:
raw = [payload.get('action')] if payload.get('action') else []
if not isinstance(raw, list):
return error_response(ErrorCodes.VALIDATION_ERROR,
'actions must be a list of action names')
actions = []
for entry in raw:
name = (entry or '').strip() if isinstance(entry, str) else ''
if name and name not in actions:
actions.append(name)
if not actions:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Select at least one action to hold')
if not reason:
return error_response(ErrorCodes.VALIDATION_ERROR,
'A hold requires a reason explaining why')
for name in actions:
if not is_allowed_action(name):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Unknown or disallowed action: {name!r}')
# Only desired-mode actions are holdable. Accepting anything else would
# write a row that suppresses nothing, so the UI would report a hold that
# has no effect - worse than refusing. Checked for every action up front so
# the request is rejected whole rather than half-applied.
holdable = {entry['action'] for entry in HOLDABLE_ACTIONS_FOR(asset_id)}
unholdable = [name for name in actions if name not in holdable]
if unholdable:
return error_response(
ErrorCodes.VALIDATION_ERROR,
f'Not under desired-state enforcement on this endpoint, so there '
f'is nothing to hold: {", ".join(sorted(unholdable))}')
minutes = payload.get('minutes')
try:
minutes = int(minutes) if minutes is not None else None
except (TypeError, ValueError):
return error_response(ErrorCodes.VALIDATION_ERROR,
'minutes must be a whole number')
# Zero is legal and means indefinite: the hold stands until released by
# hand. Negatives are not - they would place an already-expired hold, which
# reads as success in the UI while suppressing nothing.
if minutes is not None and minutes < 0:
return error_response(ErrorCodes.VALIDATION_ERROR,
'minutes cannot be negative (use 0 for an '
'indefinite hold)')
indefinite = minutes == 0
window = 'indefinite' if indefinite else minutes
audit_log(
action='policyholdplaced',
entitytype='Asset',
entityid=asset_id,
changes={
'actions': actions,
'reason': reason,
'minutes': window,
'indefinite': indefinite,
'placedby': getattr(current_user, 'username', None),
},
details=(f'Policy hold placed on {", ".join(actions)} '
f'({window}): {reason}'),
)
for name in actions:
placed = PLACE_POLICY_HOLD(
assetid=asset_id,
action=name,
reason=reason,
userid=getattr(current_user, 'userid', None),
username=getattr(current_user, 'username', None),
minutes=minutes,
)
if not placed:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Policy holds are unavailable on this site')
return success_response(ACTIVE_POLICY_HOLDS(asset_id))
@endpoints_bp.route('/<int:asset_id>/policyholds/<action>', methods=['DELETE'])
@require_permission('endpoints.command')
def release_endpoint_policy_hold(asset_id: int, action: str):
"""End a hold early, resuming reconciliation of one action.
Admin-only on the same reasoning as placing one: releasing a hold turns
policy enforcement back on for a machine someone may still be working on,
which is the same authority in the opposite direction.
"""
if not caller_is_role('admin'):
return error_response(ErrorCodes.FORBIDDEN,
'Releasing a policy hold requires the admin role',
http_code=403)
released = RELEASE_POLICY_HOLD(asset_id, action)
if not released:
return error_response(ErrorCodes.NOT_FOUND,
f'No active hold on {action!r} for this endpoint',
http_code=404)
audit_log(
action='policyholdreleased',
entitytype='Asset',
entityid=asset_id,
changes={
'action': action,
'releasedby': getattr(current_user, 'username', None),
},
details=f'Policy hold released early on {action}',
)
return success_response(ACTIVE_POLICY_HOLDS(asset_id))
@endpoints_bp.route('/<int:asset_id>/lock', methods=['GET'])
@require_permission('endpoints.view')
def get_endpoint_lock(asset_id: int):
"""The endpoint lock standing on this endpoint, or null.
Read-gated rather than admin-gated, and that is the point of the feature: an
operator whose command was refused needs to see "locked by <admin>: mid
repair" and self-serve. Hiding the lock from non-admins would leave them
with a bare 403 and a ticket to file about a machine that is working fine.
"""
lock = ACTIVE_LOCK_FOR(asset_id)
return success_response(lock.to_dict() if lock else None)
@endpoints_bp.route('/<int:asset_id>/lock', methods=['POST'])
@require_permission('endpoints.command')
def place_endpoint_lock(asset_id: int):
"""Quarantine this endpoint against non-admin commands.
Admin-only via caller_is_role rather than current_user.hasrole, matching the
override and hold routes: a scoped personal token is a narrower caller than
its owner, so a leaked agent token must not be able to quarantine machines.
A reason is required. The refusal shown to a blocked operator quotes it
verbatim, so an empty one produces a dead end - the operator learns only
that they may not act, not who to ask or when it lifts.
"""
if not caller_is_role('admin'):
return error_response(ErrorCodes.FORBIDDEN,
'Locking an endpoint requires the admin role',
http_code=403)
payload = request.get_json(silent=True) or {}
reason = (payload.get('reason') or '').strip()
if not reason:
return error_response(ErrorCodes.VALIDATION_ERROR,
'A lock requires a reason explaining why')
lock = PLACE_ENDPOINT_LOCK(
assetid=asset_id,
reason=reason,
userid=getattr(current_user, 'userid', None),
username=getattr(current_user, 'username', None),
)
audit_log(
action='endpointlocked',
entitytype='Asset',
entityid=asset_id,
changes={
'reason': reason,
'lockedby': getattr(current_user, 'username', None),
},
details=f'Endpoint locked: {reason}',
)
return success_response(lock.to_dict())
@endpoints_bp.route('/<int:asset_id>/lock', methods=['DELETE'])
@require_permission('endpoints.command')
def release_endpoint_lock(asset_id: int):
"""Lift the quarantine on this endpoint.
Admin-only on the same reasoning as setting one: unlocking returns a machine
someone may still be physically working inside to the whole operator pool,
which is the same authority in the opposite direction.
"""
if not caller_is_role('admin'):
return error_response(ErrorCodes.FORBIDDEN,
'Unlocking an endpoint requires the admin role',
http_code=403)
lock = RELEASE_ENDPOINT_LOCK(
asset_id,
userid=getattr(current_user, 'userid', None),
username=getattr(current_user, 'username', None),
)
if lock is None:
return error_response(ErrorCodes.NOT_FOUND,
'No active lock on this endpoint',
http_code=404)
audit_log(
action='endpointunlocked',
entitytype='Asset',
entityid=asset_id,
changes={
'lockedby': lock.lockedbyusername,
'reason': lock.reason,
'releasedby': getattr(current_user, 'username', None),
},
details=f'Endpoint unlocked (was locked: {lock.reason})',
)
return success_response(None)
@endpoints_bp.route('/<int:asset_id>/commands', methods=['POST'])
@require_permission('endpoints.command')
def enqueue_endpoint_command(asset_id: int):
"""Queue an allowlisted command for one endpoint.
Rejects unknown actions and refuses to stack a second pending/running
command on the same endpoint, so the queue stays at-most-one-in-flight and
the WebUI toggle cannot race itself.
Rejects unknown actions, refuses non-admin callers while an endpoint lock
stands, and refuses to stack a second pending/running command on the same
endpoint, so the queue stays at-most-one-in-flight and the WebUI toggle
cannot race itself.
"""
payload = request.get_json(silent=True) or {}
action = (payload.get('action') or '').strip()
@@ -2123,6 +2396,24 @@ def enqueue_endpoint_command(asset_id: int):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'Unknown or disallowed action: {action!r}')
# Endpoint lock: a quarantine on the CALLER axis, checked before the
# permission and policy gates below so the refusal can name the lock, its
# owner and its reason. Answering with a generic policy denial instead would
# send an operator hunting through Configuration Policy for a cause that is
# not there - the machine is fine, someone has simply reserved it.
#
# Admins pass straight through with no override argument and no reason. The
# lock gates WHO may act, not WHICH actions exist, so there is no policy
# decision here to break past; requiring break-glass would mean an admin had
# to override their own team's quarantine to do the repair it was set for.
lock = ACTIVE_LOCK_FOR(asset_id)
if lock is not None and not caller_is_role('admin'):
return error_response(
ErrorCodes.FORBIDDEN,
f'Endpoint locked by {lock.lockedbyusername or "a deleted user"}: '
f'{lock.reason}',
http_code=403)
# Destructive actions (for example system_restart) demand a stricter grant
# ON TOP OF endpoints.command, so a remote reboot cannot ride the routine
# toggle permission.
@@ -2138,10 +2429,59 @@ def enqueue_endpoint_command(asset_id: int):
f'Action {action!r} requires the {elevated} permission',
http_code=403)
# Break-glass: an admin may push past a Configuration Policy denial, but
# only by asking for it explicitly and saying why. The override is scoped to
# POLICY gating alone - the elevated permission check above still stands, so
# this cannot conjure a script run for someone who lacks endpoints.script.
# It widens WHERE an action may run, never WHICH actions the caller may use.
override = bool(payload.get('policyoverride'))
overridereason = (payload.get('overridereason') or '').strip()
if action not in ALLOWED_ACTIONS_FOR(assetid=asset_id):
return error_response(
ErrorCodes.VALIDATION_ERROR,
f'Action not permitted by policy for this endpoint: {action!r}')
if not override:
return error_response(
ErrorCodes.VALIDATION_ERROR,
f'Action not permitted by policy for this endpoint: {action!r}')
# caller_is_role, not current_user.hasrole: a scoped personal token is a
# narrower caller than its owner, so a leaked agent token must not
# inherit break-glass from the admin who minted it.
if not caller_is_role('admin'):
return error_response(
ErrorCodes.FORBIDDEN,
'Overriding a policy denial requires the admin role',
http_code=403)
if not overridereason:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'An override requires overridereason explaining why')
# Audited before the command is enqueued: if the enqueue fails after
# this point, the record of the attempt still survives. A break-glass
# that is only logged on success hides exactly the cases worth reviewing.
# Structured context goes in changes (JSON-serialized by the model);
# details is a plain Text column, so it carries the readable summary.
audit_log(
action='policyoverridden',
entitytype='Asset',
entityid=asset_id,
changes={
'action': action,
'reason': overridereason,
'overriddenby': getattr(current_user, 'username', None),
},
details=f'Policy denial overridden for {action}: {overridereason}',
)
# A gate-mode override is a one-shot; a desired-mode override is a fight
# with the reconciler, which would re-enqueue the opposite command on the
# next check-in and make the override look broken. Hold it open.
PLACE_POLICY_HOLD(
assetid=asset_id,
action=action,
reason=overridereason,
userid=getattr(current_user, 'userid', None),
username=getattr(current_user, 'username', None),
)
# Generic actions (service_* / process_*) must carry the subject to act on;
# fixed-target actions (mtconnect_*) must not. Validate both directions so a