Self-heal endpoint command queue wedged by a dead in-flight command

The command queue is at-most-one-in-flight, but nothing ever expired a stuck command. When an agent claimed a command (RUNNING) then died before reporting a result - killed, uninstalled, or offline mid-flight - the row stayed in-flight forever and every operator action for that endpoint returned 409 'already in flight'. Reap in-flight rows past a staleness ceiling (15m for RUNNING, 60m for PENDING) to a new STATUS_EXPIRED at enqueue time, inside the same transaction, so the queue self-heals lazily the next time someone queues work. No background job and no schema migration - status is free-text.

plugins/endpoints/models/endpoint.py

@@ -124,11 +124,24 @@ class EndpointCommand(db.Model):
 
     # Lifecycle: pending -> running -> succeeded|failed. A command is claimed
     # (pending -> running) by exactly one agent poll, then finalized by its
-    # result post.
+    # result post. A command that is claimed but whose agent never reports back
+    # (agent killed/uninstalled/offline mid-flight) would otherwise pin the
+    # at-most-one guard forever; the enqueue path reaps such rows to STATUS_EXPIRED
+    # so a dead in-flight command cannot wedge the endpoint's command queue.
     STATUS_PENDING = 'pending'
     STATUS_RUNNING = 'running'
     STATUS_SUCCEEDED = 'succeeded'
     STATUS_FAILED = 'failed'
+    STATUS_EXPIRED = 'expired'
+
+    # In-flight staleness ceilings. A RUNNING command is expected to finish in
+    # seconds to a couple of minutes; the generous ceiling tolerates a slow
+    # app install while still freeing the queue when the agent is gone. A
+    # PENDING command is claimed within one poll (~25s) by a live agent, so a
+    # long-unclaimed row means the agent is offline and the queue should not
+    # stay wedged behind it.
+    RUNNING_STALE_SECONDS = 15 * 60
+    PENDING_STALE_SECONDS = 60 * 60
 
     endpointcommandid = db.Column(db.Integer, primary_key=True)