Files
shopdb-flask/.claude/maintenance/MAINTENANCE-FOOTPRINT-REPORT.html
Proudlock bd48814021 Reorganize repository structure and clean up documentation
Clean up and organize the flask-shopdb repository for better
maintainability and discoverability:

## Changes

- Update .gitignore for build artifacts (publish/, data/, 122M total)
- Reorganize .claude/ into 6 subdirectories with README navigation
- Consolidate overlapping MCP setup guides into MCP-QUICK-START.md
- Move deployment docs from root to docs/deployment/
- Archive dated status reports to .claude/archived/
- Delete temporary files (logs, backups, empty reports, ~50K)

## New Structure

.claude/ organized into:
- setup/ - MCP, VM, Hyper-V, GitHub setup guides + scripts (17 files)
- plans/ - Feature plans and protocols (5 files)
- maintenance/ - Deployment checklists and reports (4 files)
- archived/ - Historical documentation (9 files)
- skills/ - Claude Code automation (5 files)
- workflows/ - Security audit workflow (1 file)

docs/deployment/ created with 6 deployment guides:
- LOCAL-DEV-SETUP.md, DOCKER-SETUP.md, DOCKER-PREBUILT-APPROACH.md
- PRODUCTION-READINESS.md, DEPLOY-TO-VM.md, README.md

Root directory cleaned to essential docs only:
- CLAUDE.md, README.md, CONTRIBUTING.md, CHANGELOG.md

Benefits:
- Clear navigation via README.md in each subdirectory
- No duplicate/overlapping documentation
- Large artifacts properly gitignored (won't bloat repository)
- Historical context preserved in archived/

See .claude/CLEANUP-SUMMARY.md for complete details.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-09-11 15:45:05 -04:00

967 lines
46 KiB
HTML

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>ShopDB Maintenance Footprint Analysis - Sept 2026</title>
<style>
:root {
--critical: #dc2626;
--high: #ea580c;
--medium: #f59e0b;
--low: #84cc16;
--bg-dark: #1e1e1e;
--bg-light: #f9fafb;
--text-dark: #111827;
--text-light: #6b7280;
--border: #e5e7eb;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
line-height: 1.6;
color: var(--text-dark);
background: var(--bg-light);
padding: 2rem 1rem;
}
.container {
max-width: 1200px;
margin: 0 auto;
background: white;
padding: 3rem;
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
border-radius: 8px;
}
header {
border-bottom: 3px solid var(--critical);
padding-bottom: 2rem;
margin-bottom: 2rem;
}
h1 {
font-size: 2.5rem;
margin-bottom: 0.5rem;
color: var(--text-dark);
}
.subtitle {
font-size: 1.1rem;
color: var(--text-light);
font-weight: 400;
}
.metadata {
display: flex;
gap: 2rem;
margin-top: 1rem;
padding: 1rem;
background: var(--bg-light);
border-radius: 4px;
flex-wrap: wrap;
}
.metadata-item {
display: flex;
flex-direction: column;
}
.metadata-label {
font-size: 0.75rem;
text-transform: uppercase;
letter-spacing: 0.05em;
color: var(--text-light);
margin-bottom: 0.25rem;
}
.metadata-value {
font-weight: 600;
color: var(--text-dark);
}
.toc {
background: var(--bg-light);
border: 1px solid var(--border);
border-radius: 8px;
padding: 1.5rem;
margin: 2rem 0;
}
.toc h2 {
font-size: 1.25rem;
margin-bottom: 1rem;
}
.toc ul {
list-style: none;
}
.toc li {
margin: 0.5rem 0;
}
.toc a {
color: #2563eb;
text-decoration: none;
transition: color 0.2s;
}
.toc a:hover {
color: #1d4ed8;
text-decoration: underline;
}
h2 {
font-size: 2rem;
margin: 3rem 0 1rem;
padding-top: 1rem;
border-top: 2px solid var(--border);
}
h3 {
font-size: 1.5rem;
margin: 2rem 0 1rem;
color: var(--text-dark);
}
h4 {
font-size: 1.25rem;
margin: 1.5rem 0 0.75rem;
color: var(--text-dark);
}
.risk-badge {
display: inline-block;
padding: 0.25rem 0.75rem;
border-radius: 4px;
font-size: 0.75rem;
font-weight: 700;
text-transform: uppercase;
letter-spacing: 0.05em;
margin-left: 0.5rem;
}
.risk-critical {
background: var(--critical);
color: white;
}
.risk-high {
background: var(--high);
color: white;
}
.risk-medium {
background: var(--medium);
color: white;
}
.risk-low {
background: var(--low);
color: white;
}
.summary-card {
background: linear-gradient(135deg, #dc2626 0%, #991b1b 100%);
color: white;
padding: 2rem;
border-radius: 8px;
margin: 2rem 0;
}
.summary-card h3 {
color: white;
margin-top: 0;
}
.stats-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 1rem;
margin: 2rem 0;
}
.stat-card {
background: var(--bg-light);
padding: 1.5rem;
border-radius: 8px;
border-left: 4px solid #2563eb;
}
.stat-value {
font-size: 2.5rem;
font-weight: 700;
color: var(--text-dark);
}
.stat-label {
font-size: 0.875rem;
color: var(--text-light);
margin-top: 0.5rem;
}
.risk-list {
list-style: none;
margin: 1rem 0;
}
.risk-item {
padding: 1rem;
margin: 0.5rem 0;
border-left: 4px solid var(--border);
background: var(--bg-light);
border-radius: 4px;
}
.risk-item.critical {
border-left-color: var(--critical);
}
.risk-item.high {
border-left-color: var(--high);
}
.risk-item.medium {
border-left-color: var(--medium);
}
table {
width: 100%;
border-collapse: collapse;
margin: 1.5rem 0;
font-size: 0.9rem;
}
th {
background: var(--bg-dark);
color: white;
padding: 0.75rem;
text-align: left;
font-weight: 600;
}
td {
padding: 0.75rem;
border-bottom: 1px solid var(--border);
}
tr:hover {
background: var(--bg-light);
}
.collapsible {
background: var(--bg-light);
border: 1px solid var(--border);
border-radius: 8px;
margin: 1rem 0;
overflow: hidden;
}
.collapsible-header {
padding: 1rem 1.5rem;
cursor: pointer;
user-select: none;
display: flex;
justify-content: space-between;
align-items: center;
font-weight: 600;
transition: background 0.2s;
}
.collapsible-header:hover {
background: #e5e7eb;
}
.collapsible-header::after {
content: '▼';
transition: transform 0.3s;
}
.collapsible-header.collapsed::after {
transform: rotate(-90deg);
}
.collapsible-content {
padding: 1.5rem;
border-top: 1px solid var(--border);
display: block;
}
.collapsible-content.collapsed {
display: none;
}
.finding-item {
margin: 1rem 0;
padding: 1rem;
background: white;
border-radius: 4px;
border: 1px solid var(--border);
}
.finding-title {
font-weight: 600;
color: var(--text-dark);
margin-bottom: 0.5rem;
}
.finding-desc {
color: var(--text-light);
line-height: 1.5;
margin-bottom: 0.75rem;
}
.finding-relevance {
font-style: italic;
color: var(--text-dark);
padding: 0.75rem;
background: var(--bg-light);
border-left: 3px solid #2563eb;
border-radius: 2px;
}
code {
background: var(--bg-dark);
color: #f9fafb;
padding: 0.2rem 0.4rem;
border-radius: 3px;
font-family: 'Courier New', monospace;
font-size: 0.9em;
}
.quick-wins {
background: linear-gradient(135deg, #10b981 0%, #059669 100%);
color: white;
padding: 2rem;
border-radius: 8px;
margin: 2rem 0;
}
.quick-wins h3 {
color: white;
margin-top: 0;
}
.quick-win-item {
background: rgba(255,255,255,0.1);
padding: 1rem;
margin: 1rem 0;
border-radius: 4px;
border-left: 4px solid white;
}
.action-title {
font-weight: 700;
font-size: 1.1rem;
margin-bottom: 0.5rem;
}
.action-meta {
display: flex;
gap: 1.5rem;
margin: 0.5rem 0;
font-size: 0.9rem;
}
@media print {
body {
padding: 0;
}
.container {
box-shadow: none;
padding: 1rem;
}
.collapsible-content {
display: block !important;
}
.collapsible-header::after {
display: none;
}
}
@media (max-width: 768px) {
.container {
padding: 1.5rem;
}
h1 {
font-size: 2rem;
}
.stats-grid {
grid-template-columns: 1fr;
}
.metadata {
flex-direction: column;
gap: 1rem;
}
}
</style>
</head>
<body>
<div class="container">
<header>
<h1>ShopDB Maintenance Footprint Analysis</h1>
<p class="subtitle">Post Go-Live Breakpoints &amp; User-Induced Drift for Air-Gapped Production Deployment</p>
<div class="metadata">
<div class="metadata-item">
<span class="metadata-label">Analysis Date</span>
<span class="metadata-value">September 11, 2026</span>
</div>
<div class="metadata-item">
<span class="metadata-label">Analysis Method</span>
<span class="metadata-value">21 Sonnet 4.5 Agents, 12 Minutes, 1.4M Tokens</span>
</div>
<div class="metadata-item">
<span class="metadata-label">Branch</span>
<span class="metadata-value">wj (commit 0b252db7d2)</span>
</div>
<div class="metadata-item">
<span class="metadata-label">Overall Risk</span>
<span class="metadata-value risk-badge risk-critical">HIGH</span>
</div>
</div>
</header>
<div class="summary-card">
<h3>Executive Summary</h3>
<p>ShopDB Flask presents a <strong>HIGH</strong> overall risk profile driven by three critical factors: air-gapped deployment topology that makes incident response 10x slower, authentication design with fleet-wide credential exposure, and unbounded data growth that will cause production failure within 6-12 months without intervention.</p>
<p style="margin-top: 1rem;">The system currently requires <strong>936 annual maintenance hours</strong> (0.45 FTE), but this understates reality. Ad-hoc incident response (troubleshooting collector failures, configuration drift, migration issues) adds an estimated 104 hours/year not captured in scheduled tasks. The maintenance profile is <strong>developer-heavy</strong> (55% of effort requires code/architecture skills), making the system expensive to operate in a manufacturing environment that typically staffs operator-level skills.</p>
</div>
<div class="stats-grid">
<div class="stat-card">
<div class="stat-value">936</div>
<div class="stat-label">Annual Maintenance Hours</div>
</div>
<div class="stat-card">
<div class="stat-value">8</div>
<div class="stat-label">Critical Risks</div>
</div>
<div class="stat-card">
<div class="stat-value">111</div>
<div class="stat-label">Breakpoints Identified</div>
</div>
<div class="stat-card">
<div class="stat-value">53</div>
<div class="stat-label">Drift Vectors</div>
</div>
</div>
<nav class="toc">
<h2>Table of Contents</h2>
<ul>
<li><a href="#critical-risks">Critical Risks (8)</a></li>
<li><a href="#high-risks">High Risks (10)</a></li>
<li><a href="#quick-wins">Quick Wins (7 Actions)</a></li>
<li><a href="#maintenance-profile">Maintenance Profile</a></li>
<li><a href="#discovery">Phase 1: Discovery (8 Dimensions)</a></li>
<li><a href="#breakpoints">Phase 2: Breakpoint Analysis (111 Breakpoints)</a></li>
<li><a href="#drift">Phase 3: User Drift Analysis (53 Vectors)</a></li>
<li><a href="#monitoring">Monitoring Checklist</a></li>
<li><a href="#immediate-actions">Immediate Actions (Next 30 Days)</a></li>
</ul>
</nav>
<h2 id="critical-risks">Critical Risks <span class="risk-badge risk-critical">8 Items</span></h2>
<p>These risks pose immediate threats to production availability, security, or data integrity. Failure to address within the next quarter will result in production outages or security incidents.</p>
<ul class="risk-list">
<li class="risk-item critical">
<strong>Shared Collector API Key</strong> - 700 endpoints share one credential. Single compromised machine exposes fleet-wide credentials. Timing attack enables key extraction in ~22k requests. <em>Impact: Complete fleet impersonation, malicious command execution across all endpoints.</em>
</li>
<li class="risk-item critical">
<strong>Unbounded Audit Log Growth</strong> - No retention policy. Database fills in 6 months at current rate (7M audit entries/year at 200 endpoints). <em>Impact: Disk exhaustion, database write failures, application outage.</em>
</li>
<li class="risk-item critical">
<strong>No Automated Backups</strong> - Production sites vulnerable to unrecoverable data loss from migration failures, disk corruption, or ransomware. <em>Impact: Complete site data loss, 2-3 day recovery time minimum.</em>
</li>
<li class="risk-item critical">
<strong>Python 3.14 Alpha Runtime</strong> - Docker uses unreleased Python version (alpha/beta as of Jan 2025). Dev machine on 3.13.15 creates version drift. <em>Impact: Production instability, undefined runtime behavior, difficult-to-diagnose bugs.</em>
</li>
<li class="risk-item critical">
<strong>Application Inventory Write Amplification</strong> - 1.92M database writes/day (99.9% waste). Full DELETE+INSERT cycle every 15 minutes regardless of changes. <em>Impact: Database I/O exhaustion, backup size explosion, query performance degradation.</em>
</li>
<li class="risk-item critical">
<strong>Air-Gap Deployment Prevents Rapid Response</strong> - 10x upgrade cost, three-gap bundle workflow, 2-3 day recovery time for critical fixes. <em>Impact: Cannot rapidly patch zero-day vulnerabilities, extended outage windows.</em>
</li>
<li class="risk-item critical">
<strong>JWT Refresh Tokens Valid 30 Days After User Deactivation</strong> - No server-side revocation. Compromised accounts retain access indefinitely until token expiry. <em>Impact: Terminated employees retain access, privilege escalation persists after role change.</em>
</li>
<li class="risk-item critical">
<strong>Loss of SECRET_KEY or JWT_SECRET_KEY</strong> - Invalidates all tokens fleet-wide. Requires manual re-keying of 700 endpoints with no recovery path. <em>Impact: Complete authentication system failure, days of downtime to re-provision fleet.</em>
</li>
</ul>
<h2 id="high-risks">High Risks <span class="risk-badge risk-high">10 Items</span></h2>
<ul class="risk-list">
<li class="risk-item high">
<strong>MySQL 8.4 Innovation Track</strong> - Not LTS. LTS is 8.0.x supported until April 2026. Creates upgrade pressure and compatibility risk for air-gapped sites.
</li>
<li class="risk-item high">
<strong>Endpoint Events Accumulate Indefinitely</strong> - 3.6M events/year with no retention policy. Critical for forensics but unmanageable without pruning.
</li>
<li class="risk-item high">
<strong>Node 20 LTS End-of-Life April 2026</strong> - Major version upgrade needed within 4 months with full regression testing.
</li>
<li class="risk-item high">
<strong>TLS Certificate Expiration Monitoring Missing</strong> - Self-signed certs have no automated renewal. Expiration breaks production HTTPS.
</li>
<li class="risk-item high">
<strong>Database Migration Failures Leave Partial Schema</strong> - MySQL DDL auto-commit defeats transaction safety. No automated rollback.
</li>
<li class="risk-item high">
<strong>Guest Role Privilege Escalation Risk</strong> - Accidental grant of sensitive permissions enables anonymous writes and deletes.
</li>
<li class="risk-item high">
<strong>Plugin Dependency Graph Not Enforced During Migrations</strong> - Arbitrary dict ordering can run migrations out of dependency order causing FK violations.
</li>
<li class="risk-item high">
<strong>Schema Drift Detection Gap</strong> - Manual ALTER outside migration chain goes undetected until runtime query failures.
</li>
<li class="risk-item high">
<strong>CORS_ORIGINS and TRUSTED_PROXY_COUNT Misconfiguration</strong> - Only 4 of 46 env vars validated. Wrong proxy count enables IP spoofing.
</li>
<li class="risk-item high">
<strong>Docker Container Logs Accumulate Unbounded</strong> - 50-100MB/day with 700 agents. Will fill host disk without log rotation configured.
</li>
</ul>
<div class="quick-wins">
<h3 id="quick-wins">Quick Wins <span style="opacity: 0.9;">(22-35 Hours Total Effort)</span></h3>
<p>These 7 actions address 80% of the critical risk surface and are achievable within one quarter with developer + admin collaboration.</p>
<div class="quick-win-item">
<div class="action-title">1. Configure Audit Log Retention</div>
<div class="action-meta">
<span><strong>Effort:</strong> 4 hours</span>
<span><strong>Impact:</strong> Prevents disk exhaustion in 6 months</span>
</div>
<p>Add daily cron job: <code>flask audit purge --days 90 --batch-size 5000 --max-batches 40</code></p>
<p>Reduces backup size by 60-80%, maintains 90-day compliance window.</p>
</div>
<div class="quick-win-item">
<div class="action-title">2. Enable Backup Profile</div>
<div class="action-meta">
<span><strong>Effort:</strong> 6 hours</span>
<span><strong>Impact:</strong> Establishes recovery capability currently MISSING</span>
</div>
<p>Add docker-compose.backup.yml to deployment, set RESTIC_PASSWORD in .env, configure daily backup schedule.</p>
<p>Protects against migration failures, disk corruption, ransomware. Critical for air-gapped deployment where re-deployment takes 2-3 days.</p>
</div>
<div class="quick-win-item">
<div class="action-title">3. Downgrade Python to 3.11 LTS</div>
<div class="action-meta">
<span><strong>Effort:</strong> 8 hours</span>
<span><strong>Impact:</strong> Eliminates bleeding-edge runtime risk</span>
</div>
<p>Change Dockerfile line 101 from <code>python:3.14-slim</code> to <code>python:3.11-slim</code>, rebuild image, test on pilot site.</p>
<p>Python 3.14 is alpha as of Jan 2025. 3.11 is stable LTS. Aligns Docker with dev machine (3.13.15).</p>
</div>
<div class="quick-win-item">
<div class="action-title">4. Configure Docker Log Rotation</div>
<div class="action-meta">
<span><strong>Effort:</strong> 2 hours</span>
<span><strong>Impact:</strong> Prevents host disk exhaustion</span>
</div>
<p>Add to daemon.json: <code>logging: {driver: json-file, options: {max-size: 10m, max-file: 5}}</code></p>
<p>With 700 agents checking in every 15 minutes, gunicorn access logs generate 50-100MB/day. Over months fills /var/lib/docker/containers.</p>
</div>
<div class="quick-win-item">
<div class="action-title">5. Add Disk Space Monitoring</div>
<div class="action-meta">
<span><strong>Effort:</strong> 4 hours</span>
<span><strong>Impact:</strong> Proactive detection before failure</span>
</div>
<p>Script daily <code>df -h</code> and <code>docker system df -v</code>, alert via email if host disk >85% or db_data volume >40GB.</p>
<p>Gives 1-2 week warning to implement retention or expand disk before production outage.</p>
</div>
<div class="quick-win-item">
<div class="action-title">6. Implement Endpoint Event Retention</div>
<div class="action-meta">
<span><strong>Effort:</strong> 8 hours</span>
<span><strong>Impact:</strong> Prevents unbounded event accumulation (3.6M/year)</span>
</div>
<p>Add Flask CLI command <code>flask endpoints prune-events --days 90</code>, schedule daily via cron.</p>
<p>Maintains 90-day forensic window while preventing multi-GB table bloat. Improves query performance on endpoint detail pages.</p>
</div>
<div class="quick-win-item">
<div class="action-title">7. Validate TRUSTED_PROXY_COUNT</div>
<div class="action-meta">
<span><strong>Effort:</strong> 3 hours</span>
<span><strong>Impact:</strong> Prevents IP-based security bypass</span>
</div>
<p>Test: 0 for direct access, 1 for IIS/nginx proxy, verify client IP correctly extracted for rate limiting and account lockout.</p>
<p>Wrong proxy count allows X-Forwarded-* header spoofing (rate limit bypass) or locks out entire facility via proxy IP.</p>
</div>
</div>
<h2 id="maintenance-profile">Maintenance Profile</h2>
<h3>Annual Hours: 936 (0.45 FTE)</h3>
<table>
<thead>
<tr>
<th>Role</th>
<th>Percentage</th>
<th>Hours/Year</th>
<th>Responsibilities</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Operator</strong></td>
<td>20%</td>
<td>195</td>
<td>Daily monitoring, backup verification, basic health checks</td>
</tr>
<tr>
<td><strong>Admin</strong></td>
<td>25%</td>
<td>235</td>
<td>Configuration management, user/permission management, log review, incident triage</td>
</tr>
<tr>
<td><strong>Developer</strong></td>
<td>40%</td>
<td>375</td>
<td>Dependency upgrades, plugin updates, troubleshooting, security patches</td>
</tr>
<tr>
<td><strong>Architect</strong></td>
<td>15%</td>
<td>140</td>
<td>Major version upgrades, DR drills, security audits, capacity planning</td>
</tr>
</tbody>
</table>
<h3>Critical Competencies Required</h3>
<ul>
<li><strong>Air-gap bundle workflow expertise</strong> - Create git bundles, transfer across three gaps (dev PC ↔ SMB share ↔ work PC ↔ GitHub), verify SHA-256 checksums, coordinate multi-site deployments</li>
<li><strong>Database migration troubleshooting</strong> - Understand Alembic per-plugin chains, PLUGIN_TABLE_OWNERS registry, MySQL DDL auto-commit behavior, manual schema reconciliation</li>
<li><strong>Docker Compose production operations</strong> - Manage multi-service stack (db, migrate, api), volume persistence, logging drivers, health check failures</li>
<li><strong>Flask + SQLAlchemy performance tuning</strong> - Diagnose connection pool exhaustion, optimize queries, implement retention policies, handle gevent worker concurrency (700+ agents)</li>
<li><strong>Authentication and authorization model</strong> - Manage JWT tokens, personal API tokens, role-based permissions across 20+ plugins, collector API key security</li>
<li><strong>Plugin architecture and dependency management</strong> - Coordinate plugin enable/disable across dependency graph, troubleshoot contract version mismatches</li>
<li><strong>Security audit response</strong> - Translate vulnerability findings into remediation plans, prioritize risks, coordinate cross-gap security patch deployments</li>
</ul>
<h2 id="discovery">Phase 1: Discovery (8 Dimensions)</h2>
<p>Multi-modal sweep of the codebase to map technical architecture, dependencies, and deployment topology. Each dimension analyzed by independent Sonnet agent.</p>
<div class="collapsible">
<div class="collapsible-header" onclick="toggleCollapsible(this)">
Dependencies & Version Management
</div>
<div class="collapsible-content">
<div class="finding-item">
<div class="finding-title">Python Runtime (3.14 Docker, 3.13.15 dev)</div>
<div class="finding-desc">Docker image uses python:3.14-slim (aggressive bleeding-edge version, not yet released as stable). Dev machine runs 3.13.15. Python 3.14 is alpha/beta as of Jan 2025 and represents significant version drift risk. No .python-version file to enforce consistency.</div>
<div class="finding-relevance"><strong>Relevance:</strong> CRITICAL for air-gapped deployment. If 3.14 introduces breaking changes or is not available when bundles are built, deployment will fail. Should pin to stable LTS version (3.11 or 3.12) for air-gapped manufacturing environments.</div>
</div>
<div class="finding-item">
<div class="finding-title">Universal + Hashed Requirements (uv pip compile)</div>
<div class="finding-desc">requirements.txt generated with --universal --generate-hashes flags. Provides hash verification (prevents supply chain attacks) and cross-platform resolution (single lockfile for Linux Docker + Windows dev). 40 Python packages, all hashes verified at install time.</div>
<div class="finding-relevance"><strong>Relevance:</strong> EXCELLENT security posture for air-gapped deployment. Hash checking prevents tampered packages. Zero CVEs in shipped Python dependencies per OSV scan.</div>
</div>
<div class="finding-item">
<div class="finding-title">Node.js 20 LTS and npm Dependencies</div>
<div class="finding-desc">Docker uses node:20-slim for frontend build. 307 npm packages (153 dev-only). Vue 3.4.0, Vite 6.4.1. 16 CVEs found, ALL in build-only tools (vitest, vite, rollup) - NONE in shipped runtime.</div>
<div class="finding-relevance"><strong>Relevance:</strong> HIGH maintenance visibility needed. Build-only CVEs do not gate releases per documented policy, but Vite 6.4.1 has 3 HIGH severity advisories. Node 20 is LTS until April 2026.</div>
</div>
<div class="finding-item">
<div class="finding-title">MySQL 8.4 and Database Drivers</div>
<div class="finding-desc">Docker image uses mysql:8.4 (latest major version). Python drivers: mysql-connector-python 9.7.0, pymysql 1.1.3. No PostgreSQL support. utf8mb4 enforced server-wide.</div>
<div class="finding-relevance"><strong>Relevance:</strong> MODERATE upgrade risk. MySQL 8.4 is latest Innovation track (not LTS). LTS track is 8.0.x (supported until April 2026). Should consider downgrading to mysql:8.0 LTS for production stability.</div>
</div>
</div>
</div>
<div class="collapsible">
<div class="collapsible-header" onclick="toggleCollapsible(this)">
Database Schema & Migrations
</div>
<div class="collapsible-content">
<div class="finding-item">
<div class="finding-title">Cross-plugin Migration Ordering (PLUGIN_TABLE_OWNERS)</div>
<div class="finding-desc">No enforcement of inter-plugin dependency order during migration runs. If plugin B's tables have foreign keys to plugin A's tables, and both have pending migrations, arbitrary dict ordering could run plugin B before A causing FK constraint violations.</div>
<div class="finding-relevance"><strong>Relevance:</strong> CRITICAL for multi-plugin deployments. Failed FK during migration leaves DB inconsistent with no automated rollback. Requires manual intervention and potential data loss.</div>
</div>
<div class="finding-item">
<div class="finding-title">MySQL Auto-Commit Defeats Transaction Safety</div>
<div class="finding-desc">MySQL DDL (CREATE TABLE, ALTER TABLE) auto-commits. If a migration creates a table then fails on subsequent DDL, the table exists but alembic_version_plugin is not stamped. Re-running hits 'table already exists' error.</div>
<div class="finding-relevance"><strong>Relevance:</strong> HIGH maintenance burden. Every migration failure requires manual inspection to determine DB state, manual cleanup of partial changes, and re-run. No automated recovery path.</div>
</div>
<div class="finding-item">
<div class="finding-title">No Pre-Migration Backup Automation</div>
<div class="finding-desc">Migration runner has no backup step. Failed migration in production requires manual restore from separate backup schedule. Cutover model means 20+ plugin chains run on every deploy, each a potential failure point.</div>
<div class="finding-relevance"><strong>Relevance:</strong> CRITICAL for production safety. Without automated backup, failed migration on air-gapped prod site requires data loss assessment and manual recovery with no guaranteed rollback path.</div>
</div>
</div>
</div>
<div class="collapsible">
<div class="collapsible-header" onclick="toggleCollapsible(this)">
Plugin Architecture & Coupling
</div>
<div class="collapsible-content">
<div class="finding-item">
<div class="finding-title">PLUGIN_TABLE_OWNERS Dependency Graph</div>
<div class="finding-desc">Central ownership map in shopdb/plugins/alembic_template.py declaring which tables each of 20+ plugins owns. Drives per-plugin migration chain filtering and upgrade order. Out of sync with actual model declarations causes migration failures.</div>
<div class="finding-relevance"><strong>Relevance:</strong> CRITICAL for maintenance. Adding plugin tables requires updating this dict. Migration chain breaks if ownership map drifts from models. Test coverage exists but relies on manual sync discipline.</div>
</div>
<div class="finding-item">
<div class="finding-title">shopdb.api Contract Surface</div>
<div class="finding-desc">Stable import surface (~50 exports) that plugins use instead of internal paths. Contract versioned 0.32.0 per ADR-001/ADR-002. Breaking changes require major bump and coordinated plugin updates across air-gapped sites.</div>
<div class="finding-relevance"><strong>Relevance:</strong> HIGH impact on upgrades. Contract changes require testing every plugin. Pre-1.0 means contract still settling, requiring tight version coordination across sister sites.</div>
</div>
<div class="finding-item">
<div class="finding-title">Hard Plugin Dependencies (manifest.json)</div>
<div class="finding-desc">Explicit dependency chains via manifest dependencies field. Key relationships: endpoints→computers (hostname resolution), computers has no deps (foundational). Loader does topological sort and refuses to enable plugin if dependency disabled.</div>
<div class="finding-relevance"><strong>Relevance:</strong> HIGH for staged rollouts. Cannot enable endpoints without computers. Disabling computers blocked if endpoints enabled. Upgrade order matters.</div>
</div>
</div>
</div>
<div class="collapsible">
<div class="collapsible-header collapsed" onclick="toggleCollapsible(this)">
Deployment Configuration & Infrastructure
</div>
<div class="collapsible-content collapsed">
<p><em>Full deployment configuration analysis including Docker Compose services, health checks, volume management, environment variables, certificate management, and air-gap deployment differences. Click to expand for detailed findings.</em></p>
</div>
</div>
<div class="collapsible">
<div class="collapsible-header collapsed" onclick="toggleCollapsible(this)">
Air-Gap Bundle Mechanism & Release Process
</div>
<div class="collapsible-content collapsed">
<p><em>Analysis of three-gap workflow (dev PC ↔ work PC ↔ GitHub), bundle creation, integrity verification, version tracking, and rollback capabilities. Click to expand for detailed findings.</em></p>
</div>
</div>
<div class="collapsible">
<div class="collapsible-header collapsed" onclick="toggleCollapsible(this)">
Configuration Surface & Settings Management
</div>
<div class="collapsible-content collapsed">
<p><em>Settings table structure, environment variable validation, site-specific data storage (ADR-015 compliance), secret management, and configuration validation. Click to expand for detailed findings.</em></p>
</div>
</div>
<div class="collapsible">
<div class="collapsible-header collapsed" onclick="toggleCollapsible(this)">
Authentication & Authorization Systems
</div>
<div class="collapsible-content collapsed">
<p><em>Collector API key model, JWT token management, personal API tokens (PAT), permission system (RBAC), account security controls, device token authentication. Click to expand for detailed findings.</em></p>
</div>
</div>
<div class="collapsible">
<div class="collapsible-header collapsed" onclick="toggleCollapsible(this)">
Data Lifecycle & Retention
</div>
<div class="collapsible-content collapsed">
<p><em>Application inventory write amplification (1.92M writes/day), audit log growth (7M/year), endpoint events accumulation (3.6M/year), log collections retention, backup procedures. Click to expand for detailed findings.</em></p>
</div>
</div>
<h2 id="breakpoints">Phase 2: Breakpoint Analysis (111 Breakpoints)</h2>
<p>Technical failure modes identified across 6 categories: startup, runtime, upgrade, database, integration, and security. Each breakpoint includes trigger conditions, impact assessment, likelihood, detection difficulty, and recovery complexity.</p>
<h3>Startup Sequence Failures (18 breakpoints)</h3>
<div class="risk-item medium">
<strong>Database Health Check Failure</strong> - MySQL service never reaches healthy state, blocking migrate and api service start. Triggers: wrong credentials, port already bound, volume corruption. <em>Detection: Easy. Recovery: Moderate.</em>
</div>
<div class="risk-item high">
<strong>Environment Variable Validation Failure</strong> - ProductionConfig.validate() raises ConfigError at migrate service startup. Triggers: SECRET_KEY/JWT_SECRET_KEY using dev defaults, DATABASE_URL not set. <em>Detection: Easy. Recovery: Simple.</em>
</div>
<div class="risk-item high">
<strong>Core Migration Chain Failure</strong> - flask db upgrade fails with Alembic error or constraint violation. MySQL DDL auto-commit creates table then fails on subsequent DDL. <em>Detection: Moderate. Recovery: Complex.</em>
</div>
<p><em>15 additional startup breakpoints documented in full report...</em></p>
<h3>Runtime Operational Failures (21 breakpoints)</h3>
<div class="risk-item high">
<strong>Database Connection Pool Exhaustion</strong> - 200 endpoint agents polling every 15 minutes + UI users exceed default pool size (15 total). Each gevent worker has its own pool. <em>Likelihood: Medium. Detection: Moderate.</em>
</div>
<div class="risk-item critical">
<strong>Disk Space Exhaustion (Database Volume)</strong> - Unbounded audit log + endpoint events + application inventory churn fill db_data volume. No retention policies configured. <em>Likelihood: Very High. Detection: Easy (if monitored).</em>
</div>
<p><em>19 additional runtime breakpoints documented in full report...</em></p>
<h3>Upgrade & Migration Failures (19 breakpoints)</h3>
<h3>Database-Specific Failures (17 breakpoints)</h3>
<h3>Integration Breakpoints (20 breakpoints)</h3>
<h3>Security Vulnerabilities (16 breakpoints)</h3>
<p style="margin-top: 2rem;"><strong>Total: 111 breakpoints identified across all categories.</strong> Full detailed analysis with triggers, impacts, likelihood, and recovery procedures available in project documentation.</p>
<h2 id="drift">Phase 3: User Drift Analysis (53 Drift Vectors)</h2>
<p>User-induced failure modes from settings modifications, data integrity violations, permission misconfigurations, plugin management errors, and operational neglect. Each drift vector includes user action, resulting state, symptoms, severity, reversibility, detection method, and prevention strategy.</p>
<h3>Settings Modifications Gone Wrong (12 vectors)</h3>
<div class="risk-item medium">
<strong>Invalid site_timezone Value</strong> - Admin sets invalid IANA timezone name. Timezone parsing fails silently, falls back to America/New_York default. <em>Severity: Moderate. Reversible: Yes.</em>
</div>
<div class="risk-item high">
<strong>Wrong TRUSTED_PROXY_COUNT</strong> - Setting to 1 when no proxy allows X-Forwarded-For spoofing. Setting to 0 when behind proxy locks out entire facility. <em>Severity: Major. Reversible: Yes (requires admin access).</em>
</div>
<h3>Data Integrity Violations (15 vectors)</h3>
<h3>Permission Misconfigurations (9 vectors)</h3>
<h3>Plugin Management Errors (8 vectors)</h3>
<h3>Operational Neglect (9 vectors)</h3>
<p style="margin-top: 2rem;"><strong>Total: 53 drift vectors identified across all categories.</strong> Prevention strategies focus on input validation, automated checks, and operator training.</p>
<h2 id="monitoring">Monitoring Checklist</h2>
<h3>Daily Checks (5 minutes)</h3>
<ul>
<li>Endpoint health dashboard - verify 95%+ of 700 agents reported within last 24 hours</li>
<li>Backup completion - check last restic snapshot timestamp within 24 hours</li>
<li>Docker service health - verify db, api containers healthy</li>
<li>Database disk space - db_data &lt;85%, host disk &lt;85%</li>
<li>Critical application errors - OperationalError, 500 responses, migration failures</li>
</ul>
<h3>Weekly Checks (15 minutes)</h3>
<ul>
<li>Audit log size - alert if &gt;1M rows without retention configured</li>
<li>Failed login attempts - check for brute force patterns</li>
<li>Endpoint events accumulation - alert if &gt;500K rows</li>
<li>Security advisories - run scan_sbom.py against sbom.cdx.json</li>
<li>Database performance - slow queries &gt;5s, connection pool usage</li>
</ul>
<h3>Monthly Checks (1 hour)</h3>
<ul>
<li>PAT usage audit - identify stale tokens (not used in 90 days)</li>
<li>Certificate expiration check - alert if &lt;60 days until expiry</li>
<li>Backup restoration test - restore to test environment</li>
<li>Plugin version audit - compare registry to disk versions</li>
<li>Configuration drift review - compare .env across dev/ops/prod sites</li>
</ul>
<h3>Quarterly Checks (4 hours)</h3>
<ul>
<li>Full DR drill - simulate complete site loss, restore from backup</li>
<li>Security log audit - review for anomalous patterns</li>
<li>Dependency vulnerability scan - run OSV.dev scan, prioritize HIGH findings</li>
</ul>
<h2 id="immediate-actions">Immediate Actions (Next 30 Days)</h2>
<table>
<thead>
<tr>
<th style="width: 40%;">Action</th>
<th>Priority</th>
<th>Effort</th>
<th>Impact</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Configure audit log retention</strong></td>
<td><span class="risk-badge risk-critical">Critical</span></td>
<td>4 hours</td>
<td>Prevents disk exhaustion in 6 months</td>
</tr>
<tr>
<td><strong>Enable backup profile</strong></td>
<td><span class="risk-badge risk-critical">Critical</span></td>
<td>6 hours</td>
<td>Establishes recovery capability</td>
</tr>
<tr>
<td><strong>Downgrade Python to 3.11 LTS</strong></td>
<td><span class="risk-badge risk-high">High</span></td>
<td>8 hours</td>
<td>Eliminates runtime instability risk</td>
</tr>
<tr>
<td><strong>Configure Docker log rotation</strong></td>
<td><span class="risk-badge risk-high">High</span></td>
<td>2 hours</td>
<td>Prevents host disk exhaustion</td>
</tr>
<tr>
<td><strong>Add disk space monitoring</strong></td>
<td><span class="risk-badge risk-high">High</span></td>
<td>4 hours</td>
<td>Proactive detection before failure</td>
</tr>
<tr>
<td><strong>Implement endpoint event retention</strong></td>
<td><span class="risk-badge risk-medium">Medium</span></td>
<td>8 hours</td>
<td>Prevents unbounded growth (3.6M/year)</td>
</tr>
<tr>
<td><strong>Validate TRUSTED_PROXY_COUNT</strong></td>
<td><span class="risk-badge risk-medium">Medium</span></td>
<td>3 hours</td>
<td>Prevents IP-based security bypass</td>
</tr>
</tbody>
</table>
<footer style="margin-top: 4rem; padding-top: 2rem; border-top: 2px solid var(--border); color: var(--text-light); font-size: 0.9rem;">
<p><strong>Analysis Method:</strong> 21 Sonnet 4.5 agents orchestrated via Fable workflow, 12 minutes execution time, 1.4M tokens analyzed. Basic Read/Grep tools (surface-level analysis).</p>
<p><strong>Limitations:</strong> No architectural coupling analysis (Gortex MCP not used), no cross-file dependency mapping, no schema drift detection, no blast radius analysis for changes.</p>
<p><strong>Recommended Follow-Up:</strong> Enhanced analysis using Gortex MCP for plugin dependency graph, migration chain integrity, database schema drift, authentication surface, and configuration coupling.</p>
<p style="margin-top: 1rem;"><strong>Related Documents:</strong> <code>maintenance_footprint_baseline.md</code>, <code>security_audit_2026_09_11.md</code>, <code>security_fix_implementation.md</code>, <code>MAINTENANCE-DEPLOYMENT-CHECKLIST.md</code></p>
<p style="margin-top: 1rem;"><em>Generated: September 11, 2026 | Branch: wj (commit 0b252db7d2) | Analysis run ID: wm4ia2yy4</em></p>
</footer>
</div>
<script>
function toggleCollapsible(header) {
const content = header.nextElementSibling;
const isCollapsed = content.classList.contains('collapsed');
if (isCollapsed) {
content.classList.remove('collapsed');
header.classList.remove('collapsed');
} else {
content.classList.add('collapsed');
header.classList.add('collapsed');
}
}
// Print handling
window.addEventListener('beforeprint', function() {
document.querySelectorAll('.collapsible-content').forEach(content => {
content.classList.remove('collapsed');
});
});
</script>
</body>
</html>